A case study of student development across project-based learning units in middle school chemistry

Abstract

Numerous theoretical and empirical studies have claimed that project-based learning (PBL) exerts a positive impact on student development. This study explores the development and changes of students across project-based learning units by qualitative research methods. We followed a student group (1 boy and 3 girls) from one class in grade 9 over 3 time points within 1 school year. Classroom observation, focus group student interviews and artifact collection were used to collect data at the end of three units over time.

Qualitative research methods were employed for data analysis to determine what competencies students demonstrate and how these competencies changed during 3 units. The results revealed that this student group demonstrated both cognitive (e.g., understanding of core ideas, use of scientific practices, problem solving and creativity) and non-cognitive competencies (e.g., motivation to learn chemistry, collaboration, environmental awareness and perseverance). Three competencies (understanding of core ideas, motivation to learn chemistry, and collaboration) were shown in all three units, and these three competencies gradually improved as the units progressed. The across project-based learning units showed a promising effect on student development. This study concludes with a discussion of challenges and promises for using across project-based learning units to support student development.

Introduction

Students in the twenty-first century live in an interrelated, diversified and rapidly changing world. Economic, social, cultural, digital, demographic, environmental and epidemiological forces shape young people’s lives, and young people face unprecedented opportunities and challenges (OECD, 2019). This generation should be equipped with scientific literacy and some necessary skills to cope with these challenges. To adapt our children to the life of the global community in the twenty-first century, we should substantially alter our way of education for students (Sawyer, 2014). Project-based learning cultivates students’ curiosity and builds an understanding of core ideas in science, enabling students to solve problems and become responsible citizens with scientific literacy (Krajcik & Czerniak, 2018).

Students’ meaningful understanding is built over time, therefore, it will take time to provide many opportunities for students to learn disciplinary core ideas, crosscutting concepts, science and engineering practices (National Research Council, 2012). Researchers suggested that a longer duration of experience in PBL helps foster students’ cognitive competencies (such as knowledge and skill) and non-cognitive competencies (such as motivation and interest of learning science) (Bhuyan et al., 2020; Jenkins, 2017). Several studies have shown the value of using units that develop across time by building upon previous understanding and experiences (Krajcik et al., 2008; Roseman et al., 2008). However, just a few studies (Fortus et al., 2015; Margel et al., 2008; Shin et al., 2019) have demonstrated the value of using coherent curriculum materials across grades. Shin et al. (2019) proved that students who experience a coherent PBL curriculum build a deeper understanding of atomic structure over time, particularly in high- and middle-performing schools. More studies need to be conducted on the long-term impacts on students when they are immersed in the PBL approach (Jenkins, 2017).

In China, under the pressures of senior high school entrance examinations and college entrance examinations, very few schools implement multiple PBL units in one semester. In 2018, Beijing Huai Rou Number 1 Middle School and our team set up a “Project-based Learning Program (PBLP)” using project-based learning instead of traditional chemical teaching in 9th grade, which is a milestone for China’s project-based learning. In this program, we continued to focus on the students’ development across project-based learning units.

PBL increases the development of both learners’ knowledge and skills (Krajcik & Czerniak, 2018; Barak & Raz, 2000; Hasni et al., 2016). Artifacts show what students have learned (Krajcik & Blumenfeld, 2006; Krajcik & Shin, 2014), and teachers can use artifacts to know how students’ understanding develops across various units in PBL (Krajcik & Shin, 2014). However, in most cases, the artifacts were assessed limited to the artifacts themselves, such as product design and product quality (Chua et al., 2014; Torres et al., 2019), rather than the development of students’ key competencies. It is unclear what competencies students demonstrate as they develop artifacts in a PBL environment. By tracking the learning process of one student group in different units, this study attempted to identify the competencies that students demonstrate across the units as well as the competencies levels in PBL.

Literature review

The impact of PBL on students

Project-based learning is more effective than traditional learning approaches in science education (Ayaz & Söylemez, 2015). Scholars believe that PBL promotes the development of students’ multi-dimensional competencies, including cognitive dimension, emotional attitude dimension and social skills (Barak & Raz, 2000; Hasni et al., 2016).

PBL promotes the development of students’ cognitive dimension

Scholars are particularly interested in the development of students’ cognitive dimension in PBL. On the one hand, researchers believe that PBL can help students develop a meaningful understanding of disciplinary core ideas and improve their academic performance (Santyasa et al., 2020; Harris et al., 2015; Rivet & Krajcik, 2004; Geier et al., 2008; Marx et al., 2004; Williams & Linn, 2003). Moreover, PBL can promote the development of higher-order competencies related to students’ science learning, such as problem solving (Hong et al., 2012; Kokotsaki et al., 2016; Mettas & Constantinou, 2008), problem raising (Irit et al., 2018), argumentation (Hsu et al., 2016), critical thinking (Holmes & Hwang, 2016; Irit et al., 2018), creativity (Hanif et al., 2019; Storer, 2018), and collaborative problem solving (Lavonen et al., 2002).

Disciplinary core ideas

Disciplinary core ideas, also known as big ideas, are essential ideas of a discipline, which can be used to explain many phenomena, and as tools to explore more complex phenomena and solve problems, they are also the cornerstones for in-depth study of a discipline (Stevens et al., 2009). Students participated in the project-based science curriculum outperformed those in the comparison curriculum in understanding disciplinary core ideas in science (Harris et al., 2015; Hong et al., 2012). Students engaged in PBL units understood the concepts deeply, but these results are unlikely to be captured in the standardized tests used to measure science achievement (Prince & Felder, 2006). Assessment in a project-based learning classroom is a continuous process that is embedded in instruction (Krajcik & Czerniak, 2018). Zhao et al. (2019) developed a framework to evaluate students’ understanding of core ideas in chemistry according to their performance of presentation for artifacts in a project-based class. The study found that, students established understanding of the conception (such as “combustion”) in a unit, but it is difficult to establish understanding of the big idea (such as “chemical change”). Establishing understanding of big ideas may require multiple units.

PBL promotes the development of students’ emotional dimension

For the development of the emotional dimension, researchers have also conducted many empirical studies in PBL. For example, PBL can improve students’ motivation (Filippatou & Kaldi, 2010; Holmes & Hwang, 2016), interest and engagement in learning (Bencze & Bowen, 2009; Hugerat et al., 2004; Hung et al., 2012; Vaino et al., 2012), enhance students’ learning attitude (Kanter & Konstantopoulos, 2010; Toolin, 2004), improve their self-efficacy (Clark, 2014), self-esteem (Cook et al., 2012; Kilinc, 2010), and develop their attitude and enthusiasm for science (Barak, 2004; Tseng et al., 2013).

Motivation

Motivation is an activation and intention that drives and maintains a person’s action, and makes the action achieve a certain goal. People can be motivated by different types of factors (Ryan & Deci, 2000). Several factors contribute to a learner’s motivation, including self-efficacy, intra-personal attribution, and anxiety (Holmes & Hwang, 2016). Improving students’ motivation for scientific learning, stimulating students’ interest, and increasing learning engagement are important aspects of education. In project-based learning, benefit of the high degree of personal participation of children, students will have strong autonomy in exploring issues related to daily life (Baines et al., 2017; Condliffe et al., 2017). Moreover, engaging in science and engineering are useful for stimulating students’ curiosity, attracting their interest and motivating them to pursue learning (National Research Council, 2012).

Carrabba and Farmer (2018) found significant differences in students’ motivation levels before and after PBL and direct instruction. Increasing student intrinsic motivation and engagement in the classroom is addressed through autonomy, competence, relatedness, and relevance (Sackstein, 2017). Ostroff (2016) stated that motivation comes from the genuine curiosity that is part of every human’s consciousness. Bi (2019) developed a inventory of chemistry learning motivation based on self-determination theory, and classified motivation into 6 levels according to the classical taxonomy of educational objectives in the affective domain (Bloom et al., 1964). Bi found that (a) PBL can improve students’ motivation to learn chemistry, different types of units have different effects on students’ motivation, and (b) students’ motivation to learn chemistry increased more after teachers’ teaching practice been improved.

PBL promotes the development of students’ social skills

PBL has also been found to develop students’ social skills, strengthen group collaboration and improve students’ interpersonal skills (Williams & Simon, 2017; Xu & Liu, 2010; Lee et al., 2015). When students successfully learn how to better collaborate with one another, their intra-group process and the intra-individual learning process may be more effectively guided in acquiring knowledge (Dawes & Sams, 2004; Littleton & Miell, 2004).

Collaboration

Collaboration is critical for twenty-first century, and it is increasingly sought after in education (Bentley & Cazaly, 2015). Collaboration is a coordinated and synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem (Roschelle & Teasley, 1995). In PBL environment, all members of the groups will collaborate with each other. To promote collaboration, teachers will help students develop collaborative ability, supporting students in learning how to discuss ideas with each other, use scientific evidence to defend their ideas and work in small groups. Learners develop their understanding of disciplinary core ideas by sharing and discussing ideas with others (Blumenfeld et al., 1996). Studies have shown that students benefit from small-group learning (Slavin, 1996; Wenzel, 2000; Williamson & Rowe, 2002). Students who work in collaborative groups with other students are more motivated and successful than those who do not do this, especially in reasoning and critical thinking skills (Wenzel, 2000).

Most previous studies have provided evidence that PBL has positive effects on student development. There are two main data sources for these empirical studies. One is the pre- and post-test data of students, and the other is to use techniques to collect data, such as questionnaires, interviews, classroom observation, and student logs. Most studies are quantitative research on the learning effect of one unit through pre- and post-tests (Carrabba & Farmer, 2018; Filippatou & Kaldi, 2010; Harris et al., 2015; Tseng et al., 2013; Xu & Liu, 2010). Some researchers have conducted qualitative analysis on students’ performances during a unit (Hong et al., 2012; Hanif et al., 2019; Williams & Simon, 2017), while other studies have used evaluation tools to track students’ performance in a 2 ~ 3 years PBL (Harris et al., 2019; Marx et al., 2004; Shin et al., 2019). Little research has been done on continuous qualitative studies of same students across different units in PBL.

Conceptual framework

Definition and features of PBL

PBL is a form of situated learning that is based on constructivism research (Lave & Wenger, 1991). Students use a collaborative approach to design solutions to real and meaningful problems in the real world in order to acquire knowledge and skills (Buck Institute for Education, 2008; Gijbels et al., 2005; Petrosino, 2004). In project-based science learning, students are engaged in real, meaningful problems that are important to them and mirror what scientists do. A project-based science classroom allows students to explore phenomena, investigate questions, discuss their ideas, engage in scientific practices, challenge the ideas of others, try out new ideas, and construct and revise models (Krajcik & Shin, 2014).

Project-based learning of PBLP meets the following six key features (Blumenfeld et al., 1991; Krajcik et al., 1994; Krajcik & Czerniak, 2013): (1) They start with a driving question. (2) They focus on learning goals for which students are required to demonstrate mastery on key science standards and assessments. (3) Students explore the driving question by participating in scientific practices. (4) Students, teachers, and community members are engaged in collaborative activities to find solutions to the driving question. (5) When engaged in science practices, students are scaffolded with learning technologies that help them participate in activities normally beyond their ability. (6) Students create a set of tangible products that address the driving question.

Analytical framework of students’ competencies

Understanding of core ideas

“The properties and transformation of substances” is one of the important concept for students to learn in chemistry, and it is also a core idea of the Compulsory Education Chemistry Course Standard(CECCS) in China (Ministry of Education of People’s Republic of China, 2012). Understanding the idea of “The properties and transformation of substances” specifically refers to knowing the main physical and chemical properties of common substances (e.g., carbon dioxide, common metals, acids and alkalies), using these properties to achieve the separation, purification and transformation of substances.

This research focuses on students’ understanding and development of “the properties and transformation of substances.” We ranked students’ understanding of this core idea into six levels according to the taxonomy in the cognitive domain (Bloom & Krathwohl, 1956) (See Table A1 in the Additional file 1).

Motivation to learn chemistry

Krathwohl, Bloom and Masia’s taxonomy of educational objectives in the affective domain is the classical theory in the field of education (Gable & Wolf, 1993; Klopfer, 2010; Laforgia, 2010). Bi’s motivation inventory has been verified to have good reliability and validity, and this inventory was specially developed for chemistry project-based learning (Bi, 2019). Based on Krathwohl, Bloom and Masia’s taxonomy of educational objectives, and Bi’s chemistry motivation inventory, we describe 5 levels of students’ motivation to learn chemistry (See Table A2 in the Additional file 1).

Collaboration

Collaboration in PBL is not ordinary cooperation, rather, it is reflected in the process of solving problems. The Assessment and Teaching of twenty-first Century Skills (ATC21S) project (Griffin et al., 2012) defined ways of measuring individual person skills in collaborative problem solving (CPS) and has been cited by many studies in the field of education (Camacho-Morles, Slemp, Oades, Morrish, & Scoular, 2019; Dieu et al., 2018; Pöysä-Tarhonen et al., 2018). One of the evaluation dimension of CPS is collaboration, which is consistent with collaboration in PBL. We adopted the description and level division of collaboration in ATC21S’s CPS framework (Hesse et al., 2015), which divides collaboration into 6 levels from lower to higher (See Table A3 in the Additional file 1).

PBL curriculum in PBLP

At Beijing Huai Rou Number 1 Middle School, the whole year chemistry curriculum in 9th grade was delivered through PBL, with all units following Project-Based Teaching Experiment Textbook: Chemistry (PBTETC) (Wang et al., 2018). There are 8 units in the textbook, which were implemented in two semesters. Each unit focuses on learning goals of CECCS. For example, the Low-Carbon Actions unit corresponds to the curriculum standard of “Understanding the carbon cycle in nature, combining examples to illustrate the properties and uses of carbon dioxide, and learning how to make carbon dioxide in the laboratory.”

A good driving question elicits a desire to learn in students (Edelson, 2001), and it makes students realize that there is an important problem that genuinely needs to be solved (Reiser, 2004). As students pursue solutions to the driving question, they develop integrated understandings of core scientific ideas (NRC, 2012). The design of each unit starts with a real driving question. These questions come from real life and can stimulate students’ motivation to learn chemistry. Each unit is broken down into 3 core tasks based on driving question, and each task contains 1 ~ 3 student activities. In each activity, there are different columns to provide students with a wealth of practical activities, such as “Independent learning,” “Investigation,” “Group communication,” “Experimental inquiry,” “Design and make” and “Check progress” to guide students’ learning (a screenshot of the textbook is shown in Fig. 1). Students could create a set of artifacts in each unit, for example, in unit 4, each student group developed a poster of the “Low-Carbon Action Convention.”

Fig. 1

Screenshots of the PBTETC textbook

In the textbook of PBTETC, three units focus on developing students’ understanding of the same core idea of “the properties and transformation of substances”. These three units set up tasks of different types and situations to realize the progressive development of students’ understanding of ideas, motivation to learn chemistry and collaboration. The theory of “situated learning” holds that, when acquiring information in a meaningful environment and connecting it with previous knowledge and experience, students can develop a connection between new information and previous knowledge, thus forming conceptual understanding (Blumenfeld et al., 1991; Krajcik & Czerniak, 2013). Students need to transfer the previous knowledge and experience when solving new problem. Through the study of these three units, students can achieve a gradual and in-depth understanding of concepts of substances. The types of project tasks cover designing a solution, analyzing production, designing and conducting investigation. The situation are from familiar and simple to unfamiliar and complex. PBL helps students answer questions about the world around them, thus stimulating their curiosity and sense of engagement in exploring what is happening (Krajcik & Czerniak, 2018). We used the frameworks in Tables A1 ~ A3 in the Additional file 1 to code “project learning objectives” in the textbook to determine the expected development level in each unit (see Fig. 2). When there were different levels of coding for the same ability in the goal, we took the highest level. For example, the objectives for unit 4 “Low Carbon Action” are:

• 1. Able to illustrate the main properties and uses of carbon dioxide with examples [UCI: Comprehension (level 2)]. Understand the relationship between the properties and uses of carbon dioxide and understand the transformation of carbon dioxide from the perspective of elements [UCI: Application (level 3)].

• 2. Actively participate in chemistry learning, understand the importance of implementing low-carbon actions, and cultivate students’ civic awareness [MLC: Responding (level 2)].

• 3. Actively participate in group collaboration, share their views, collaborate to complete the group low-carbon convention [COL: Collaboration consciousness (level 3)], and be able to analyze and explain the content of the convention based on the knowledge of carbon dioxide [UCI: Analysis (level 4)].

Fig. 2

Conceptual framework

(UCI stands for understanding of core ideas; MLC stands for motivation to learn chemistry; COL stands for collaboration.)

In addition to the competencies of understanding core ideas, motivation to learn chemistry and collaboration, these three units let students solve problems through chemical experiments, cultivate students’ scientific practice skills and problem-solving abilities. Based on the existing research on the impact of PBL on students, combined with the characteristics of these three units, in this research, we focused on the following competencies: understanding of core ideas, motivation to learn chemistry, collaboration, use of scientific practices, problem solving, and creativity.

Aims of the study

In summary, previous research on PBL has mainly studied the competencies of students in one or more aspects (Tseng et al., 2013; Xu & Liu, 2010; Williams & Simon, 2017), but little research has been done on the comprehensive value of PBL. Some studies have explored the changes in students’ performance over time (Harris et al., 2019; Marx et al., 2004), but students’ specific performance in PBL is not clear. Existing qualitative analyses, with relatively short time spans, cannot describe students’ development and changes across different units. We focus on determining students’ competencies as they construct artifacts in a PBL environment, tracking the learning development of the same students in different units. The detailed research questions are as follows:

• RQ-1: What competencies do students demonstrate and develop as they construct artifacts in a PBL environment?

• RQ-2: How do these competencies develop across the units?

Methods

As noted by Eisenhardt (1989) and Yin (2014), constructing a case study is an appropriate method when there is not much known about a topic. As little research has been done on the development of and changes in the same students across units, it is more appropriate to adopt the method of case study. To conduct this case study, we used several data sources, including classroom observation, student interviews and artifacts.

The presentation of artifacts is an important part of PBL. When students introduce their work, they present all kinds of knowledge, skills and attitudes they have acquired, which provides a good opportunity for evaluation (Krajcik & Czerniak 2018). At the end of each unit in PBLP, every student group will display and report their artifacts in class. At this time, experts and researchers go to the class of Huairou No. 1 Middle School for one day of research. In the morning, class observation was conducted, and a video camera was used to record the performance of students in class. In the afternoon, we invited 4 students in the selected group for interviews. After that, we communicated with the teacher of this class.

Participants

The participants in PBLP was a four-person student group (1 boy and 3 girls) from a class (40 students) in Beijing Huai Rou Number 1 Middle School, China, and all of them were local. Students in this class went directly to the school’s high school without taking China’s Senior High School Entrance Examination after graduating from 9th grade. Therefore, they did not have the pressure of the senior high school entrance examination compared with other middle school students, and they could spend more time in project-based learning. Before 9th grade, they had not studied chemistry, and the chemistry course of this class was taught by the same teacher, Ms. Xu, a young female teacher without any experience of project-based teaching.

We selected a four-person student group based on the chemistry scores of the 9th grade entrance examination, there was a significant difference of four students’ score rank in this group. The average score rank of the group was 4/8, mid-level in the class (there were 8 student groups in this class). The information of this student group is provided in Table 1. The four members of this group were freely chosen by themselves, and they participated in the eight project-based learning units during the academic year.

Table 1 Students’ information

Summary of chemistry curriculum in PBLP

Project-Based Teaching Experiment Textbook: Chemistry was published in 2018 and has been adopted by more than 10 middle schools in Beijing, Hebei, Shandong and other regions of China, earning extensive acclaims from teachers and students. To explore students’ understanding of “the properties and transformation of substances,” we chose three units (unit 4, unit 5 and unit 7) for research.

Unit 4 Low-Carbon Action

The greenhouse effect has had a negative influence on our lives. In this context, students will raise the driving question: How can carbon dioxide be reduced in the atmosphere to achieve a low-carbon life? In this unit, students will formulate a low-carbon convention to solve this problem. The content is so closely related to real life that it could stimulate students’ interest in learning. To formulate a low-carbon convention, students use the properties of carbon dioxide to convert it into other substances, thereby reducing the content of carbon dioxide. They work together to formulate low-carbon conventions, their sense of collaboration and environmental awareness are cultivated gradually.

Unit 5 reasonable use of metal products

Metal products are commonly used in life, this unit starts with the driving question: What problems will be encountered during the use of metal products? How do we use metal products rationally? This is a real and slightly complicated task because students should use the relevant knowledge of metal properties to analyze real vacuum cups, creatively design an instruction for vacuum cups according to users’ actual needs, and compile the manual of the designed vacuum cups. As a challenging task for individuals, it needs to be completed through group collaboration. Group members should communicate in time during the design of vacuum cups and solve problems together. In the process of completing the task, students realize the application value of the knowledge related to metal properties in life. Thus, their motivation to learn chemistry will be enhanced.

Unit 7 soil improvement

The driving question of this unit are: What are the elements required for plant growth? How do you improve the soil to make plants grow better? In this unit, students need to develop an understanding of the properties of acid, alkali and salt; explain phenomena in daily life with the properties of acid, alkali and salt; use related knowledge to plant a pot of plants they like; understand the relationship between soil acidity, soil fertility and plant growth; and establish a two-way relationship between the properties of acid, alkali and salt in real life. In this unit, students participate in a series of scientific practice, which are so motivational that students’ strong interest could be stimulated. This is an unfamiliar task, so students can better realize the importance of group collaboration to solve problems and actively participate in group collaboration.

Implementation of chemistry PBL

Student learning activities

In each unit, students went through three types of lessons: Introductory lesson, process lesson and presentation lesson. In the introductory lesson, students understood the project background, appreciated the project value and became interested in project tasks. Teacher and students put forward driving questions together, identified the project objectives, teacher led students to break down and plan the project. During the process lessons, students needed to use the core ideas to solve a series of sub-questions and experienced diversified scientific practice activities, go through many rounds of problem solving process before finally solving the problems. The problem solving process can reflect students’ problem-solving competency, as well as what core chemistry knowledge has been learned and applied in this process. Students needed to collaborate during this process. Sometimes, the teacher asked the students to report this process in presentations. In presentation lesson, student groups introduced their artifacts through PPT, posters, cartoons and sitcoms in class.

For example, in unit 4 Low-carbon Actions, students investigated the effect of greenhouse before class, they felt the urgency of addressing environmental problems, and stimulated the motivation to participate in Low-carbon Actions. In the following process lessons, students determined the source and outlet of carbon dioxide by information searching and group communication, explored the nature of carbon dioxide through experiments, found ways to reduce carbon dioxide content in the atmosphere, and developed a low-carbon convention. Finally, the groups’ low-carbon conventions were displayed in the form of posters within the class (Table A4 in the Additional file 1 lists the main activities of three units).

Teacher training

As the chemistry teacher in this class had no previous experience in project-based teaching, a PBL expert group was specially set up to guide the teacher. The expert group consisted of three professors in the field of education from Beijing Normal University, one associate professor from Capital Normal University and four teaching and research staff members from Haidian Teachers’ Training School in Beijing. Before all units started, experts provided the teacher with professional training on PBL theory. During the implementation of each unit, the teachers participated in training twice. The first guidance was before the implementation of unit, the teacher introduced her teaching design, experts helped her revise teaching design. The second training occurred after the teaching of each unit, experts observed presentation class and then conducted interviews with the teacher and students, the teacher reflected on the teaching of the whole unit, and the experts gave advice for improving teaching.

Data collection

This research mainly collected data through classroom observations, student interviews and artifact collection (see Table 2). Both classroom observations and student interview data were recorded. The qualitative method was adopted in data analysis, therefore, we needed to transcribe video and audio data into words and then encode them. The main research objects were students, in the transcript, the teacher was anonymized as T, the two researchers were anonymized as R1 and R2, and four students were anonymized as S to protect their privacy. (Statement: All videos and interviews were approved by the students and the teacher.)

Table 2 Data sources

Classroom observations

During the presentation of each unit, observers went to the school to observe the performance of the student group in class. Observers include the first author, second author and project training experts of PBLP. The purpose of observation is that we are able to observe first-hand actual information about the students, to facilitate student interviews and to help teacher preferably improve teaching.

Focus group interviews

After student groups’ presentation, four students from the selected group were invited to participate in the interviews. Each interview lasted approximately 20 min, all interviews were conducted at the school. Interviewer and observers were the same individuals. A list of interview questions was developed, and each interview began with the same questions (see Table 3 for sample questions). The interviewee’s responses guided further questions.

Table 3 Interview questions

Artifacts

With the consent of the students, we collected the students’ final artifacts of each unit and conducted an in-depth analysis of their artifacts to determine their level of understanding of core ideas. Figure 3 shows examples of student artifacts.

Fig. 3

Sample screenshots from student artifacts: (a) Low Carbon Convention Poster, (b) PPT screenshot of 55-degree cup introduction and (c) Report of Unit 7

Data analysis

The framework of Strauss and Corbin (1998) was adopted to analyze data, consisting of three steps: (1) classifying data; (2) creating patterns within each data source; and (3) examining patterns among data sources. In the following, the analysis process of each step will be detailed.

Categorizing data

To respond to the research questions, we coded the students’ performance data to determine the competencies demonstrated and developed in three units. This step was done by three coders (the first author of this paper and two master’s students majoring in chemistry education). Before coding, these three coders were trained to reach a consensus on the understanding of 6 competencies (understanding of core ideas, motivation to learn chemistry, collaboration, use of scientific practices, problem solving and creativity), and the transcribed text was then sent to the coders. They marked the text that could reflect the students’ competencies and labeled them. When 3 coders had different opinions, they resolved their differences through discussion. We did not distinguish the performance of the 4 students but evaluated the overall level of the group.

In the first round of coding, we used “interpretive” codes, which require participants’ meanings to be deciphered, and were largely conserved the sake of objectivity (Miles & Huberman, 1994). A brief outline of this coding scheme is presented below:

• Understanding of core ideas: using core ideas to explain important phenomena in daily life, use evidence to support claims, and design or evaluate scientific problem solutions.

• Problem solving: the process of finding solutions to difficult and complex issues.

• Use of scientific practices: multiple ways in which students explore and make sense of the natural and design world, such as asking questions, developing and using models, planning and carrying out investigations, analyzing and interpreting data.

• Collaboration: working well as member of a group, being loyal to the group, contributing to the group.

• Motivation to learn chemistry: behaving or taking action for intrinsic or extrinsic reasons to learn chemistry.

• Creativity: the ability to transcend traditional ideas, rules, patterns, and relationships, etc., and to create meaningful new ideas, forms, methods, and interpretations, etc.

• Environmental awareness: understanding how social, economic and environmental systems interact and support life, gradually developing an energy-saving, low-carbon, green travel, and environmentally friendly lifestyle.

• Perseverance: the disposition required to maintain effort or interest in an activity in the face of difficulties encountered, the length of time or steps involved or when opposed by someone or something.

During the coding process, we found that students also showed environmental awareness and perseverance. Therefore, we added them to the coding scheme.

Creating patterns

According to the results of the first round of coding, we found that the competencies of understanding core ideas, motivation to learn chemistry and collaboration appeared in all three units. In the second step, we focused on coding the levels of these 3 competencies. The frameworks of understanding core ideas, motivation to learn chemistry and collaboration (shown in in Tables A1 ~ A3 in the Additional file 1) were used to evaluate students’ competency levels. These three coders participated in the coding. Before coding levels, these three coders carefully read the content of the evaluation framework and tried to evaluate the same short text separately. Then, they discussed the differences of the results, selected another paragraph of text to evaluate separately and discussed again until their independent scoring results were agreed upon. After that, they completed the level evaluation of all text independently. We used SPSS 20.0 to check Kappa consistency, the consistency coefficient among these three coders was 0.929, indicating that the coding of the 3 coders was highly consistent.

Here, we show some coding fragments of unit 7 to make the coding process clearer. In this unit, students planted a pot of their favorite plants in the soil, explored the relationship between soil acidity, alkalinity, soil fertility and plant growth, wrote a complete experimental report, and reporting the research result to the class. During the project presentation, the students described the following:

“When we were determining the research topic, our members proposed to study the effect of nitrogen fertilizer on plant growth. To verify the rationality of this topic, we conducted copious literature research. We consulted the literature about the impact of soil pH on plants and understood the effect of nitrogen fertilizer on plant growth. Finally, we agreed to take “the effect of nitrogen fertilizer on the same plant” as the main research topic [collaboration level 3].

After determining the topic, we discussed which plant to choose [collaboration-level 3]. Through discussion, we found that two members planted green cirrus, so we chose this plant. We looked up the internet about the growth conditions of the green cirrus, especially the pH value [understanding of core ideas in chemistry-level 3] of the soil in which this plant lives.”

……

“The biggest difficulty we had was that the pH value of the soil samples in the park was not suitable for the growth of the green cirrus. We tried to add a large amount of water to the soil but could still not obtain the right soil pH value. This problem puzzled us for a long time. Finally, we thought of using an acid-base neutralization reaction to adjust the pH value of the soil , and we made it [understanding of core ideas at chemistry level 5].”

In the after-class interview, the students said, “In this unit, with the teacher’s help, we did many experiments after class, we also searched much data according to the teacher’s tips and finally completed this experiment. We come to know that we can solve problems by experiments. We realize that chemistry is very useful for life and study, and we are full of expectations for future study in chemistry [motivation to learn chemistry-level 4]. “.

Keywords are in bold font to judge the competencies and level of the student group, and the content in “[]” is encoded by researchers. We found that, in the same unit, the same competency was coded many times, we chose the highest level as the final competency level of the student group

Examining patterns among data sources

After coding analyses, we discussed the rationality of the above competencies and development levels with all researchers of three units, and analyzed the reasons for the development and changes of students’ competencies according to the project tasks and the teacher’s instructional design.

Results

The competencies students demonstrate and develop as they constructed artifacts in a PBL environment

In this study, we encoded the performance of one student group in three units to determine the competencies students demonstrate in PBL. We obtained the results summarized in Table 4. When the student group develop artifacts in a PBL environment, they demonstrated the competencies of understanding of core ideas, motivation to learn chemistry, collaboration, use of scientific practices, problem solving, creativity, environmental awareness and perseverance.

Table 4 Results of categorizing the data

Among these competencies, these three competencies of “understanding of core idea”, “motivation to learn chemistry” and “collaboration” were demonstrated in three units (see Table 5).

Table 5 Competencies demonstrated in PBL

The development of the competencies of “understanding of core ideas, motivation to learn chemistry, and collaboration”

After the learning of three units, this student group’s understanding of “the properties and transformation of substances”, motivation to learn chemistry, and collaboration improved. The student group’s understanding of the “the properties and transformation of substances” was developed from level 3 to level 5 (see Fig. 4), their motivation to learn chemistry developed from level 2 to level 4 (see Fig. 5), and their collaboration developed from level 3 to level 5 (see Fig. 6). Next, we used students’ specific performance to describe their development.

Fig. 4

The change trend of “understanding of core ideas”

Fig. 5

The change trend of “motivation to learn chemistry”

Fig. 6

The change trend of “collaboration”

Understanding of “the properties and transformation of substances”

Students’ understanding of “the properties and transformation of substances” was gradually improved on the basis of the previous unit. In unit 4, students said “we can use the production and conversion of carbon dioxide to achieve low carbon” and they applied conservation of elements to interpret why air-conditioning temperature can reduce carbon dioxide production. However, students could only formulate a low-carbon convention from the aspect of reducing carbon dioxide production, without considering the aspect of carbon dioxide absorption. From these expressions of students, we can judge that the students’ understanding of “the properties and transformation of substances” in unit 4 reached level 3 “Application” (see Table 6).

Table 6 Students’ performances in understanding of “the properties and transformation of substances”

In unit 4, students learned to use the properties of carbonaceous substances to realize the transformation of carbonaceous substances. The context of Low-Carbon Actions was very familiar for students, and developing a low-carbon convention is a simple application-oriented task for them. In unit 5, students were not so familiar to metal products, they used vacuum flasks every day but hardly read the instruction for vacuum flasks carefully. Besides, it is not easy for students to understand the principle of the vacuum flask. Based on the study of unit 4, students could associate the properties of the substance with the characteristics of the vacuum flask in unit 5. As shown in Table 6, students said “The innermost layer is made of stainless steel, which was chosen because of its strong thermal conductivity, high temperature resistance, and corrosion resistance.” From these words, we can see that, students knew the structure of vacuum flask and used the properties of metals and alloys to interpret the structure of each part of the vacuum flask. However, the students in this group analyzed each part in isolation and did not analyze the relationship between different parts. Therefore, they only reached level 4 of understanding of core ideas.

For students, unit 7 was the most difficult and unfamiliar one of these three units. Most students in PBLP came from urban area and had few opportunities to get in touch with traditional agriculture. This unit required students to design a complete experimental plan to solve a problem, which is extremely challenging. As shown in Table 6, students said “Our experimental process was divided into five steps,” and the description of each step in the table indicated that they had formed a coherent research plan to solve the problem, this is the performance of level 5 “Synthesis.”

Motivation to learn chemistry

With the progress of the units, students’ motivation to learn chemistry had undergone the following changes (see Table 7 for details). In unit 4, students’ interest in learning chemistry was mainly due to their curiosity about chemistry experiments, they did not truly realize the value of chemistry learning (Level 2: responding). In unit 5, they felt “chemistry is very useful for life and study”, which indicated that they recognized the value of learning chemistry (Level 3: valuing). In unit 7, the students did experiments in class, they “did many experiments after class”, and they had a strong desire to learn chemistry (Level 4: organization). Teacher Xu also mentioned in her interview that “Students’ enthusiasm for learning chemistry is getting higher and higher. After the school opening the chemistry laboratory to them, many students soaked in the laboratory whenever they were free. My office is always crowded with students asking questions. There was no such scene before PBL.” However, students’ interest in chemistry had not become a part of students’ character, they had not yet reached level 5 (characterization by value or value complex) of motivation to learn chemistry.

Table 7 Students’ performances in motivation to learn chemistry

Collaboration

Through Table 8, we can sort out the changes in student collaboration. According to the student interview of unit 4, students said that, before this unit, all the work was just done by one person. In unit 4, “Other members also do work. Slowly, everyone wants to do something for group.” This indicated that the students gradually developed a sense of collaboration and reached level 3 (collaboration consciousness). In Unit 5, when one person encountered difficulties, students used network software to collaborate online on weekends, just as they said “We used QQ telephone to discuss together, and sent the PPT to the QQ group after it was completed, so that we could revise PPT together.” In this group, everyone made a certain contribution to the group, indicating that they reached level 4 (mutual contribution). In unit 7, students adopted the learning method of group collaboration throughout the project process. During the experiment, their group also encountered difficulties, but they encouraged each other to overcome difficulties, just as the student said “The four of us encouraged each other”, “We should learn from each other”. They reflected and evaluated their own and others’ performances. In this unit, students’ collaboration competence reached level 5 (valuable collaborative relationship). Teacher Xu talked about the changes of students’ collaboration: “When I first assigned the tasks in class, the students all did their own work. Now they can quickly put into communication, and this way of learning has been adopted by other subjects.”

Table 8 Students’ performances in collaboration

Discussion

Students’ competencies demonstrated in PBL

In this study, students demonstrated their competencies in the cognitive dimension, emotional and attitude dimension, and social skills in project-based learning, which is similar to the findings of existing research (Guo et al., 2020; Hasni et al., 2016). For specific competencies, consistent with existing research, we found that students demonstrated understanding of core ideas, motivation to learn chemistry, collaboration, use of scientific practices, problem solving and creativity in PBL (Hong et al., 2012; Mettas & Constantinou, 2008; Kokotsaki et al., 2016; Hanif et al., 2019; Holmes & Hwang, 2016; Filippatou & Kaldi, 2010; Williams & Simon, 2017), but environmental awareness and perseverance seldom appeared in existing project-based learning research. These two competencies also play an important role in the future development of students and should arouse the attention of researchers.

We found that not every competency was reflected in all three units. The following three reasons may have led to this result:

• (1) Perhaps some units have unique value for the development of students. For example, Low-Carbon Actions unit is closely related to environmental issues and can cultivate students’ environmental awareness. Therefore, we suggest that teachers could pay more attention to the key competencies emphasized in the existing literature as well as fully make sense of the unique value of units.

• (2) The fidelity of teaching implementation is very important for students’ development (Shin et al., 2019). In the instructional design, the teacher only paid attention to some competencies which may cause other competencies to be ignored. According to teaching design, students’ creativity and problem-solving competencies could be cultivated cross units, but these competencies were only shown in unit 5 and unit 7.

• (3) Some competencies of students could be more fully reflected in the project process, but we only collected data from students’ artifacts and interview, procedural data were missing.

The development of students’ competencies across PBL units

It is important for learners experiencing coherent curriculum to develop depth of understanding so they can effectively use their knowledge in new situations (Fortus & Krajcik, 2012; Roseman et al., 2008; Schmidt et al., 2005; Shwartz et al., 2008). PBL materials and coherent courses can promote student development (Harris et al., 2015; Shin et al., 2019). Project-Based Teaching Experiment Textbook: Chemistry was designed based on the curriculum standards of China. The difficulty of different units is progressive, which can reflect the continuity of the curriculum materials. And courses were carried out in the order of the teaching materials. The research results indicate that the teaching materials is effective for promoting students’ development.

Understanding of core ideas

Similar to the study of Shin et al. (2019), we found that students’ understanding of the core ideas of chemistry gradually deepened over time. Shin et al. (2019) were concerned about the impact of same chemical ideas on students of different grades, we explored the development of same students in a smaller time span, it is more instructive for the teachers to design the semester- or school-year curriculum. According to the learning goals in the textbook, we presupposed that students’ understanding of core ideas ranged from level 4 to level 5 and then to level 6. However, the actual development level of students ranged from level 3 to level 4 and then to level 5. The gap between preset and actual may be caused by the teacher’s teaching practice. For example, in the PBTETC textbook, we require students in a group to formulate a low-carbon convention with multiple items. However, in actual teaching practice, the teacher required each group to show only one clause in low-carbon convention so that students’ ideas could not be shown out completely.

Motivation to learn chemistry

The quality of experiences is essential for interest development, and students’ levels of interest were higher when hands-on activities were perceived more positively (Holstermann et al., 2010). These three units in this research had tasks of different difficulties to continuously stimulate students’ interest and motivation. This research shows that students’ motivation to learn chemistry can also be gradually enhanced over time, which has important implications for studying the progression of students’ motivation to learn chemistry in project-based learning. We suggest that, when designing multiple units in PBL, teachers should pay attention to the consistency within the unit and set project tasks of different difficulties to bring students a better learning experience. Driving questions are very important for stimulating students’ motivation. Students choose research questions on their own can stimulate interest more than teachers give questions to them. Therefore, teachers should master relevant teaching strategies and provide necessary guidance to students during the teaching process.

Collaboration

Student collaboration development requires tutoring by teachers (Krajcik & Czerniak, 2018). In the course of the PBL, teachers provide special training on collaboration, which helps to improve this skill.

The development of students’ competencies requires a certain process. Similar to the research by Bhuyan et al. (2020), we also find that a longer duration of experience fosters students’ knowledge and skill development as well as increased interest in PBL. This result may encourage teachers to carry out multiple units in project-based teaching. This research portrays more specific and vivid performances of students in different competencies. This study proposes three frameworks to evaluate students’ competencies, which can be used to help teachers evaluate students’ performance, as well as promote evaluation research on PBL.

It should be reminded that the participants in this study were not under the pressure of the Chinese high school entrance examination, therefore, the teacher have the courage to completely replace traditional chemistry learning with project-based learning. When other teachers are ready to implement across units in class, we suggest teachers to consider the actual situation of the school and students. Other units (except units 4, 5, and 7) may also have impact on students’ motivation and collaboration, we did not consider this impact and it can be explored in the future.

Conclusions

In this study, one student group was tracked over three units, and their learning materials were collected. Through qualitative analysis, it was found that, when student groups constructed artifacts in a PBL environment, they demonstrated the competencies of understanding core ideas, motivation to learn chemistry, collaboration, use of scientific practices, problem solving, creativity, environmental awareness and perseverance. The levels and changes of the competencies of understanding core ideas, motivation to learn chemistry and collaboration in these three units were analyzed. After the learning of three units, students’ levels of these three competencies improved, and a progressive development trend emerged. The research results have important implications for the curriculum design, implementation and evaluation of PBL.

Due to the limitations of personnel and time, only one group was selected for tracking and observing in this study. When multiple groups are selected, students’ development is more complicated. For the study of students’ general development, sample size should be expanded, and the integrity of the data should be enhanced in future research.