From implementation to reflection: exploring faculty experiences in a curriculum-focused FOLC through multi-case analysis

Abstract

Faculty development and support play an important role in improving undergraduate STEM education. Learning communities, including Faculty Online Learning Communities (FOLCs), can be a valuable resource for faculty seeking professional growth. This multi-case study examined how faculty gained value through participation in a curriculum-focused FOLC. Adopting a community of practice perspective, the value creation framework developed by Wenger, et al., was used to examine cycles of value creation for the five case study participants and identify key ways in which participation in the FOLC supported value creation.

All five case study participants shared comprehensive stories about how they gained value through participation in FOLC activities over multiple years. All initiated their value creation stories by describing pedagogical approaches they were unhappy about. As their stories unfolded, they described how they implemented new and innovative strategies to address their concerns, leading to outcomes consistent with the original pedagogical goals of the learning community. Two faculty described how the community helped them see new leadership roles for themselves within their departments, outcomes that were not anticipated at the initiation of the FOLC. Other participants spoke of a decreased feeling of professional isolation. Several features of the FOLC emerged as crucial factors in value creation, including structural elements such as regular meetings, a shared curricular focus, and leadership and networking opportunities. Importantly, study participants identified extended participation in the community and continuing feedback from multiple perspectives as drivers of reflection and growth.

These findings suggest that while short term participation in a FOLC or other faculty community may help achieve more prescribed outcomes, longer term participation supports a more emergent approach to faculty development. This research offers insights for faculty, curriculum developers, administrators, and others interested in promoting faculty development.

Introduction

Change in faculty teaching practices has been identified as an essential element of improving undergraduate STEM education (PCAST, 2012; Laursen et al., 2019). Change strategies for STEM education have been characterized as prescribed, where the desired outcome is determined in advance, or emergent, where the outcome is determined as part of the change process (Henderson et al., 2010). At the faculty level, prescribed approaches “focus on teaching or training faculty on specific new instructional techniques or ways of thinking about teaching and learning,” and assume that the change agent can effectively define the desired outcome. Indeed, many effective, university-level, STEM education practices exist (Freeman et al., 2014; Von Korff et al., 2016), but implementing these is challenging, and faculty benefit from substantial support (Henderson et al., 2007; Henderson et al., 2012).

In contrast, emergent approaches “typically focus on encouraging faculty to reflect on and improve their instruction.” Characterized as “developing reflective practitioners” (Henderson et al., 2010), emergent approaches involve participants in defining outcomes. We view emergent approaches as a necessary complement to prescribed ones. Faculty rarely use instructional strategies or practices “as is” (Foote et al., 2014); instead, they must make modifications and adaptations to accommodate local circumstances (Scanlon, et al., 2019). Reflective practice, which can support reevaluation of experiences and systematic evaluation of practice, can help faculty make such choices in skillful and effective ways. Reflective practice requires dedicated time and benefits from interactions, feedback, and support in a community (Rogers, 2002; Machost & Stains, 2023).

Supporting faculty development through participation in communities can occupy a middle space on the continuum between prescribed and emergent approaches, and is consistent with recommendations to assist faculty in implementing more effective research-based instructional strategies (RBISs) (Henderson et al., 2015). Examples of responsive communities that provide a balance of structure and flexibility and promote reflective practice include communities of practice (CoP), faculty learning communities (FLCs), and faculty online learning communities (FOLCs). CoP are “groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly” (Wenger et al., 2002). FLCs specifically support higher education faculty in effectively implementing pedagogical innovations (Cox, 2004; Elliot et al., 2016). FOLCs are geographically distributed, discipline-specific FLCs (Corrales et al., 2020; Dancy et al., 2019; Price et al., 2021). In these communities, faculty can share strategies, materials, and help adapt pedagogical strategies and curricula to each other’s unique teaching contexts and needs.

Evidence indicates that FLCs and FOLCs help faculty improve their instruction, persist in using RBIS, and promote reflective practice (Corrales et al., 2020; Price et al., 2021; Rundquist et al., 2015). To understand how a FOLC can do this, we previously identified mediating processes in a FOLC designed to support instructors of the Next Generation Physical Science and Everyday Thinking (NGPET), a physics curriculum for future elementary teachers (Price et al., 2021). These processes included troubleshooting teaching challenges, sharing information and resources, having a sounding board for ideas, exploring pedagogical concepts in the context of problems of classroom practice, and having a source of affective support (e.g., encouragement and moral support). These processes contribute to participating faculty increasing their knowledge and use of effective pedagogical techniques.

The NGPET FOLC, which was designed to support instructors in teaching with a specific curriculum and help participating faculty develop as reflective practitioners, serves as the context for the multiple case study presented in this paper. Faculty participated in this FOLC for multiple years (in many cases, four years or more). Practical implementation issues were a focus at the beginning, but in later years discussions of deeper pedagogical issues gained prominence (Martin, et al., 2022), allowing faculty to explore issues that were important to them personally or in their local context. In terms of emergent-prescribed change strategies, over several years the focus of the NGPET FOLC shifted from more prescribed to more emergent.

In this study, we are interested in the value faculty derive from participating in the NGPET FOLC community, and what forms of participation contributed to this. We hope this work will inform the design of online learning communities and our understanding of the STEM faculty change process. More broadly, we hope to address the need for more and better models for supporting faculty’s implementation of RBIS, pedagogical growth, reflective practice, and professional development.

Background and theoretical framework

We view FLCs and FOLCs as communities of practice (CoP), in which people with a common interest come together to fulfill both individual and group goals in a spirit of learning, knowledge generation and sharing, and collaboration (Wenger et al., 2002). In seeking to understand faculty experiences within a FOLC, we adopted Wenger et al.’s approach to assessing value creation in communities described by Wenger et al. (2011). This approach defines value creation as the enrichment derived from community involvement and networking, specifically in the context of social learning activities. Wenger’s framework for value creation identifies five cycles of value creation. We build on this and related work, including the methodology of Booth and Kellogg (2015), to add to the spectrum of identified values and provide insights into the types of activities, design features, and interactions that lead to value creation. Examples of values fitting these cycles have been found throughout the literature, as discussed below.

• Cycle 1 includes immediate values, resulting directly from belonging to the community which are intrinsic to the community’s activities and interactions. Examples of cycle 1 values previously found in FLCs include engagement in conversation and finding community (Booth & Kellogg, 2015; Cowan & Menchaca, 2014).

• Cycle 2 includes potential values that also result from the activities and interactions but are not immediately realized. These values can be thought of as knowledge capital, to be saved and spent at a later time. Wegner et al. (2011) further divides this into several types of capital: personal assets (human capital) consisting of useful new skills or ideas, relationships and connections (social capital), access to resources (tangible capital), collective intangible assets (reputational capital) which can lead to collective action, and transformed ability to learn (learning capital). Examples from the literature include increased confidence (Booth & Kellogg, 2015), strengthened networks (Aster et al., 2021; Cowan & Menchaca, 2014), ongoing collaboration, feedback, and post-CoP participation (Booth & Kellogg, 2015; Cowan & Menchaca, 2014; Dingyloudi et al., 2019; Guldberg et al., 2021; Jurek, 2020), and acquiring and connecting knowledge and resources (Booth & Kellogg, 2015; Cowan & Menchaca, 2014; Menchaca & Cowan, 2014).

• Cycle 3 includes applied value that reflect changes in practice. These values are often the application of values created in cycles 1 and 2 and are reflected in changes in earlier practice. Examples include changes in practice using new knowledge and greater confidence in practice (Booth & Kellogg, 2015; Cowan & Menchaca, 2014; Guldberg et al., 2021; Tomkin et al., 2019).

• Cycle 4 includes realized value. This is where the members reap the rewards of the values created in earlier cycles and see improvement in performance. Examples include student performance (Jurek, 2020; Menchaca & Cowan, 2014), development of programs (Jurek, 2020) increased collaboration (Booth & Kellogg, 2015), and career advancement (Cowan & Menchaca, 2014).

• Cycle 5 includes reframing value and redefining success at an individual or collective level. This may appear as new goals or strategies, or new metrics for evaluating success. Examples include change in attitude or perspective (Cowan & Menchaca, 2014; Menchaca & Cowan, 2014) and broader definitions of role (Booth & Kellogg, 2015).

Wenger et al.’s framework has been commonly adopted (McKellar et al., 2014) as a rigorous way of understanding and assessing value creation in a variety of contexts and especially for studying CoP. Many of these studies examined online communities of STEM educators. The communities varied in size, scope, duration of member participation, and member background. Booth and Kellogg (2015) examined four large (1,200 to over 100,000 members) online educator CoP that each drew from multiple institutions, and used interviews, based on Wenger’s framework, to develop value creation stories that were then member-checked, revised and coded according to Wenger’s five value creation cycles. Aster et al. (2021) examined a much smaller (14 member) community of experienced STEM faculty participating in a single institution’s instructional improvement initiative. Other studies (see for example Cowen & Menchaca, 2014 or Dingyloudia & Strijbos, 2015) focused on communities of more novice participants, such as students or new teachers. Although many of these communities were part of long term projects, individual members often participate in them for a limited time. Exceptions can be found in Kezar et al (2017) who specifically identified communities in which a significant number of members participated over an extended time and that had a mix of experienced and novice members. Barrett et al. (2009) documented how CoP can relieve the isolation that is often felt by both early career and more experienced faculty.

Kezar et al. (2017) examined how faculty communities can be best designed to engage faculty in STEM reform work. They conducted a comparative large-scale study of four CoPs over the course of two and a half years, examining the range and types of values created by each for its members and determining which features of the communities contributed most to the successful creation of value. Kezar specifically chose large, distributed (but not entirely virtual) STEM-focused CoPs that had published case studies examining impact on classroom, CoP participants, and institutions and which documented sustainable and lasting value to participants. Through interviews, observations, and surveys, Kezar et al. found that “philosophy and personal interactions… emerged as most important to engagement and designing for outcomes.” (Kezar et al., 2017, p 238). The key interactions specifically included peer-to-peer learning, mentoring, and follow-up opportunities. Making new connections also appeared as important, but, interestingly, some previously proposed (Wenger et al., 2002) community design factors, such as allowing for different levels of participation and nurturing a regular rhythm of events, did not appear to be of central importance in the CoPs studied by Kezar et al.: “we found the Wenger and colleagues design elements can be instructive for CoPs, but they do not capture the key elements really essential for designing for engagement in the CoPs we studied” (Kezar et al., 2017, p237).

Wenger’s framework serves as a guide to general value creation in a community of practice and a method for assessing these values. Of particular interest in communities of educators is the development of reflective practice and specifically the potential transformation of teaching practice both as an emergent outcome and as a process to arrive at that outcome. As noted by Corrales et al. (Corrales 2020), reflective practice is “an individual and collaborative process, in which one reevaluates previous experiences, events, and beliefs. This action may result in a change in one’s views of a situation and potentially lead to changes in teaching practices” (Corrales 2020, p 3). Rodgers (2002), following Dewey’s work (Dewey, 1938), provides a summary of four key features of the process of reflective practice: Reflection is a progression of experiences leading to deeper understandings and connections; It is a rigorous and systematic examination of practice; It requires interaction with others in a community; It calls for growth—both of the individual and of others. Machost and Stains (2023) also identify elements necessary for developing reflective practice, including time for reflection and community support and feedback.

Both the process and outcomes of reflective practice are well aligned with Wenger’s framework (Barrett et al., 2009). The cycle 5 values of reframing and redefining success echo the transformative endpoint of reflective practice. CoP, FLCs, and FOLCs in particular, provide exactly the environmental conditions needed for the process to develop: feedback and discussion in a trusted community with shared experiences and an extended timeline which allows for deep, ongoing reflection and transformation. The values gained in earlier cycles, networking, knowledge capital, changes in practice, and the incremental successes also support reflective practice. While discussion of reflective practice usually focuses on the classroom practice of teachers and on the benefits to their students, teachers also benefit in ways both immediately related to their professional practice and beyond (Machost & Stains, 2023).

Given the potentially transformative outcomes resulting from participating in a CoP, we investigate faculty experiences in a FOLC designed around a specific curriculum whose membership represents a broad range of backgrounds. We offer a rich, detailed look at both the values created and the specific participant experiences that contributed to the value creation. To this end, we address the following research questions:

• Research Question 1: What values do members of the NGP FOLC gain as a result of their participation? We use the Wenger 5-cycle framework to analyze the value creation that is evident in the case studies, with an added emphasis toward values that contribute toward transformative, reflective practice.

• Research Question 2: What are key ways that members engaged in the NGP FOLC and changed their professional life?

We aim to identify the key avenues of engagement and development that faculty members experience as a result of their participation in the NGP FOLC that changed their professional life, and in this way comment on the process by which participation in the FOLC contributes to value creation.

Methodology

Context and setting

The Next Generation Physical Science and Everyday Thinking (NGPET) curriculum (Goldberg et al., 2020) was adapted from the Physics and Everyday Thinking (PET) curriculum (Goldberg et al., 2008) and the Physical Science and Everyday Thinking (PSET) curriculum (Goldberg et al., 2007) to more specifically align with the Next Generation Science Standards (NGSS) framework (NGSS Lead States, 2013). All three curricula were designed around active learning strategies to be used in a course for prospective elementary teachers or as a general education course. Students work in small groups, supporting their claims with evidence and developing scientific ideas through group consensus (Goldberg, et al., 2010). Students take on more responsibility for their own learning while instructors take a more guiding and supportive role quite different from a traditional lecture-teaching role where the instructor explains the science ideas to the class. The student-centered pedagogy can thus provide instructional challenges to faculty teaching the course, especially if this role is new to them.

The NGPET FOLC was designed to support faculty as implementers of the NGPET curriculum and as reflective teachers (Price et al., 2021). The FOLC leadership team consisted of developers of the curriculum, as well as research and evaluation experts. Their main roles were to monitor the community as it evolved, addressing organizational and logistical issues, implementing additional activities and strategies as appropriate, and carry out the research and evaluation efforts. The leadership team organized the initial community (of about 55 members) into five clusters, each with 6–10 participants and two cluster leads. The clusters met bi-weekly by videoconference. Between cluster meetings, members collaborated using Slack and Google Docs. Because the FOLC members used a common curriculum, the discussions within the cluster meetings were closely related to their classroom experience. They could bring current teaching and learning concerns to the group and get practical feedback from other members (Price et al., 2021).

The cluster leads were recruited by the leadership team (in fall 2016), both because they had extensive experience teaching one of two predecessors to the NGPET curriculum prior to the beginning of the FOLC (either PET or PSET), and because they were thought to be effective cluster meeting facilitators (they were all personally known to the leadership team). In spring 2017 the cluster leads piloted the new NGPET curriculum in their own classes and met periodically with the leadership team to discuss implementation issues and plan for the summer workshop that would introduce the curriculum to the regular participants (who were selected that spring) and begin planning for the cluster meetings that were to begin that fall. The cluster leads had dual roles during cluster meetings, both as facilitators and as curriculum experts. They were asked to encourage all participants to share ideas and concerns at the meetings, especially those involving implementing NGPET, as well as bring their own prior experience with the curriculum into the discussions to help address some of the problematic implementation issues. Over the next two years the cluster leads separately met periodically to discuss facilitation strategies, challenges, and possible solutions to issues that emerged from their cluster meetings (Martin et al., 2022).

During the first year of the FOLC (2017–2018), most of the issues discussed at meetings were about logistical and practical issues, since these were the priority concerns of participants who came into the FOLC without prior experience with the curriculum (Anderson, 1997). Over the ensuing years, as members of the community became more experienced teaching the NGPET curriculum, many of the cluster discussions focused on more pedagogical issues, and both the cluster leads and other participants (who by that time had taught the curriculum themselves) learned from these discussions. The organizers also arranged additional activities to engage the community. During the second year of the FOLC (2018–2019), participants could join special content groups that focused on issues of interest to them. Examples of content group topics included writing explanations, conducting whole-class discussions, forming and maintaining effective groups, and engaging in research on science identity. These content groups met monthly and alternated with monthly cluster meetings. During the third year (2019–2020) the FOLC returned to bi-weekly cluster meetings. An NGPET FOLC virtual conference was organized in winter 2020 where members presented talks on strategies and resources they had developed to teach the curriculum. During the spring of Year 3 (2020) the Covid-19 Pandemic caused the participants to shift their teaching of NGPET from in person to online instruction. In response to this new challenge, some members of the community formed groups to create a set of resources for teaching NGPET online and offered webinars to inform the whole community. During the fourth year of the FOLC (2020–2021), since all members by now were experienced implementers of the curriculum, community members took turns leading cluster meetings on topics of interest to them. Also, after the fourth year, some community members took over responsibility for organizing and running the FOLC, essentially replacing the original leadership team.

Multi-case study design

To address the research questions, the team chose a qualitative case study approach with multi-case design to provide a rich context for interpretations and descriptions of the participants’ experiences and comparisons between them (Stake, 2006; Yin, 2018). This allowed the team to focus on a small set of active individuals in the FOLC, describe their experiences, and develop a case study for each of them. Additionally, the team could compare these cases to identify similarities and differences.

As mentioned in the Background and Theoretical Perspective section, Wenger et al.’s (2011) framework was chosen as the main tool for data collection and the main part of the analysis. Wenger et al (2011) provided two templates to help participants in a community of practice describe how their participation in the community brought value to them. The participants used one template to describe in general how they gained value from the community (called the personal value narrative), and the other to describe a specific story of value creation by following prompts aligned with the five cycles of value creation described in the previous section.

Since case studies need high interaction with the participants, the team conducted two rounds of data collection and analysis. They used seven different sources of data, all providing information to address the research questions. Table 1 shows the types of data sources, when the information was collected, and a brief description of the purpose they served in this study. The sources are numbered in the Table so that they can more easily be referred to in the following discussion. Next, the process of data collection and analysis is described, including how five members of the NGPET FOLC were selected for inclusion in the multi-case study.

Table 1 Data sources for the study

Gathering value creation stories and identifying candidates for the cases

The work reported in this paper is part of a larger project of our original research team (including authors FG, EP and SM) to study the NGPET FOLC. During the FOLC's fourth year, the team sent emails to 56 current and previous members, asking if they would be willing to share their experiences of how the FOLC provided value to them. The team followed up with a request of those who responded positively to fill out the two Wenger et al. (2011) templates, give permission to have their stories included in the broader research project, and indicate interest in joining the research team. One template, called the personal value creation narrative, prompted the members to write something about how participation in the FOLC changed them as professionals, affected their social connections, helped their professional practice, and changed their ability to influence their world as professionals, at three different stages of their involvement. The other template, called the specific value creation story, prompted the members to focus on a particular event or activity that created value for them, and trace that process through the five cycles of value creation mentioned in the Background and Theoretical Perspective section. Eleven FOLC members submitted the templates and agreed to have their work become part of the research effort (Data source I). Two (TM and MB) joined the research team, increasing its number to five (all of whom are co-authors of this paper).

At the beginning of the process of analyzing the eleven submitted specific value creation stories (the vcs), the research team had decided on the multi-case study design and began the process of developing criteria for selecting case study participants. First, three of the FOLC members who submitted their value creation stories were members of this research team (TM, SM and MB), and they were excluded as case study candidates to avoid raising any problematic methodological issues resulting from researcher and participant being the same person. Next, only five of the remaining vcs narratives suggested stories that involved all five cycles, whereas three of them did not. (Even though those three had included responses in the cycle 5 section of the Wenger template, the research team decided that those responses were more indicative of cycle 4 values, not cycle 5.) Because the team wanted to include comprehensive value creation stories that extended across all five cycles, including realized values, they decided not to include those three in the study. The remaining five were considered candidates for the case studies. Among this group, three were cluster leaders with extensive experience teaching NGPET's predecessors (PET or PSET) before joining the FOLC. Additionally, there were two regular members: one had taught PET only once prior to joining, while the other had no previous experience with the curriculum.

Conducting interviews and choosing the final cases

The research team next drafted narrative descriptions of the value creation stories for the five potential case study candidates. To try to make the narratives complete and coherent in the spirit of Wenger et al.’s (2011) description of indicators (events or activities) for each of the five cycles, the research team took the liberty to re-story parts of the narratives, to put them in a more logical framework (Creswell & Poth, 2018). The team highlighted areas in the vcs that needed clarification. They also added background information to each narrative based on survey data and additional information the participants had provided to the FOLC leadership over the duration of the FOLC (data source II). The draft case study narratives therefore included both the re-storied vcs and the additional background information.

The team next decided to interview the five potential case study candidates to get feedback on the draft case narratives to make sure they fit with the candidates’ experiences (member checking). They sent the draft case narratives to the candidates and asked their permission to be interviewed, and to schedule them. All five of the case study candidates agreed to be interviewed.

The semi-structured interviews (Merriam & Tisdell, 2016) were designed around the draft case narratives and included some additional questions about how the FOLC had been supportive in the candidates’ value creation process (Booth & Kellogg, 2015; Kezar et al., 2017). Two members of the research team jointly administered each interview, both to ensure all important questions were asked and to help with the (later) interpretation of the interviewee’s comments. During the administration of the interview the case study candidates were asked to read through each paragraph of the draft case narratives and comment on whether they agreed with what was written and to add anything relevant. Additionally, they were asked for clarifications and were posed further questions about the FOLC. Although the structure of the interview protocols (see the supplementary materials) was the same for all five case study candidates, the specific follow-up questions asked were narrative-specific (data source III).

The results from the interviews convinced the research team that they would include all five candidates in the final multi-case study: all the stories traversed all five cycles of the value creation process (Wenger et al., 2011), and the five candidates began their involvement in the curriculum focused FOLC with a wide range of prior experience teaching a previous version of the NGPET curriculum. The research team then worked on finalizing the five case narratives in three steps. First, they edited the initial draft case narratives by clarifying some of the statements and adding comments that the case study candidates made about the FOLC. Second, they expanded the initial case narratives by including additional information from the pre-FOLC interviews for the three cluster leads (data source IV), post-year 1 interviews for four of the five candidates (data source V); and transcripts of cluster meetings, etc. for all five candidates (data source VI). Third, they sent the near-final narratives back to the case study participants for final member checking and the feedback (data source VII) was then incorporated into the final version of the cases.

The five cases tell the value creation stories and background of Clay, Cruz, Courtney, Julia, and Luli (all pseudonyms). Their case narratives are included in the Findings section.

The research team then analyzed the case narratives to specifically address the two research questions. To address research question 1, the team summarized the main indicators for each cycle and organized them in a way that would make the values gained by the participants easy to compare. For research question 2, the team synthesized themes from the case narratives that addressed both the important ways that members engaged in the FOLC and also ways they changed their professional life according to the Wenger et al (2011) framework. Further, the team analyzed the case narratives to look for connections between value creation and reflective practice, building on Rodgers’ (2002) four key features of the process of reflective practice. Finally, aside from analyzing the case narratives for specific responses to the two research questions and connections with reflective practice, the team also developed additional comparative insights involving characteristics of the case participants, leadership opportunities and supportive features of the NGPET FOLC. The results of all these analyses are included in the Discussion section.

Validity and Reliability

To help ensure the internal validity or credibility of the findings, the research team used triangulation methods involving multiple sources of data and collection methods, multiple investigators, and member checking (Creswell & Poth, 2018; Merriam & Tisdell, 2016). As shown in Table 2 the team collected and analyzed data from participant-submitted personal narratives and vcs, multiple semi-structured interviews, and FOLC surveys and transcripts from various FOLC meetings. All that data was used to construct the final case narratives. All five of the authors on this paper were involved in analyzing the data and reaching agreement at various stages during the construction of the final case narratives. Furthermore, two of the researchers (FG and EP) were part of the FOLC leadership team and were involved in the design and management of the FOLC over the four years included in this study. As such, they were in a good position to validate claims made by the case study participants regarding activities they were involved in that brought them value. The other three members of the team (SM, TM and MB) were active participants in the FOLC who had also submitted value creation stories, so they were very familiar with both the FOLC activities and the VCS process, which helped in the analysis of the stories submitted by others. Angen (2000) emphasizes negotiation and dialogue with participants as an important factor in validity of the qualitative studies. In line with this, the team used two phases of member checking to increase the credibility of the findings. They sent case study participants an initial draft version of the case narratives so they could provide feedback during the vcs interviews, and they sent near final versions of the case narratives near the end of the process to obtain final feedback on the accuracy of the claims made.

Table 2 Values gained by the five case participants according to the Wenger et al. (2011) framework

To help ensure that the findings from our multi-case study might be transferable, that is, useful to readers in their own context, the researchers used two strategies. First, the case narratives used thick descriptions, that is a detailed description of the value creation stories with substantial evidence in the form of case participant quotations from interviews and meetings (Merriam & Tisdell, 2016). Second, the team was able to select the five cases that represented a range of curriculum experience prior to joining the FOLC, different roles within the community (cluster leads and regular participants), and different kinds of values acquired through participation (Merriam & Tisdell, 2016).

Reliability in qualitative research deals with the extent to which the findings are consistent with the data, helping to ensure that the study is dependable (Merriam & Tisdell, 2016). The process of triangulation, mentioned above, provides one way of ensuring that the findings are reliable. In addition, in our study researchers used a multi-step process to help ensure the final case narratives were dependable. During the first phase of data collection and analysis described above, pairs of researchers first independently constructed initial case narratives to represent the participants’ responses in the Wenger et al. (2011) template, then compared their narratives and reasonings and agreed on best narratives. These were then shared with the other researchers on the team for feedback and additional modifications, if appropriate. During the second phase of data collection and analysis, two researchers interviewed each of the candidate case study participants. The pair of researchers then collaboratively modified the existing case narrative based on the interview and presented their analysis to the rest of the research group for feedback. Members of the research team extended the case narratives by including additional information from interviews and meeting transcripts, and these additions were discussed and agreed on by the whole team. Finally, using member checking during both the middle and end of the researchers’ analysis procedures also helped ensure that the study findings were consistent with the data.

Findings

In this section we address the research questions: What values do members of the NGP FOLC gain as a result of their participation? What are keyways that members engaged in the NGP FOLC and changed their professional life? We do so by presenting the case narratives for five participants, Clay, Cruz, Courtney, Julia and Luli. The first three were recruited into the FOLC as cluster leaders, while the other two joined as regular participants. Each of these narratives includes the value creation story following the Wenger et. al. (2011) template, information about the participant's background and prior experience teaching the curriculum pedagogy, and some comments about how the FOLC supported their value creation process. The answers to the two research questions are embedded in these narratives.

Clay’s case study

Clay is a tenured faculty member at a university on the west coast that is classified by Carnegie as Master’s College and University (larger programs). He joined the NGPET FOLC at its inception in 2017 as one of the cluster leads. In his personal narrative, he wrote that his reason for participating in the FOLC was “to learn to be a better and more supportive instructor for my physical science courses, and also to help others in that same way.” Prior to joining, he had taught the NGPET curriculum for three years and had taught the predecessor PET curriculum for seven years before that. Therefore, he was very familiar with the curriculum and its pedagogical approaches before he began to facilitate cluster meetings.

In his pre-FOLC interview he talked about his approach to interacting with students. He tends to take somewhat of a hands-off approach to intervening in student discussions unless they have specific questions.

I’ve kind of learned that inserting myself isn’t usually helpful. I’m kind of monitoring and asking probing questions and making sure that everyone’s on track. … I mean I’ve taught this so many times that I feel I have a chance to try to be less interactive with the students as they are working through it. … I’m around to answer questions.

In that same interview Clay talked about how this ‘hands-off approach also applies to his reluctance to offer suggestions to colleagues in his department regarding pedagogical issues unless they specifically ask him.

We have kind of an odd culture in my department, where academic freedom is highly valued and highly protected, so definitely there is a hesitance to try to tell other people how to teach, and not step on their toes. I guess the culture is that you can ask someone for help, or suggestions or advice, but it’s not as common to just offer that advice without any prompting if that makes sense.

Later, during his post year 1 interview Clay was asked to compare the nature of the conversations he had with colleagues in his FOLC cluster versus those with his colleagues in his department. [There were a few other members of his department who were teaching NGPET at the same time, but they were not members of the FOLC.]

There's a lot of similarities. I think the FOLC conversations can get a lot deeper a lot of the time, because the local conversations, a lot of time it's thinking about pressing issues of what activity are we doing next week, and what's the exam going to look like, and just normal team-teaching kind of stuff. And, we don't always have the time to think, and talk about deeper pedagogical issues. Sometimes we do, but not always, whereas FOLC, we talk about all of that stuff all the time. … So, the depth of the FOLC conversations tend to be greater, but other than that, I think it was a lot of the same ... I mean, we all have the same issues, right? We talked about a lot of the same things.

He was also asked what aspects of his experiences in the FOLC had he enjoyed the most. He mentioned the cluster meeting discussions, but also expressed a concern he had about his role during those meetings.

Yeah, to me it's the conversations. Talking with other people about things that I'm ... I mean, in theory I'm a leader, but I'm dealing with the same issues that they are, so just having ... we're all so busy, it's not that common to get an hour to sit and talk to people about pedagogy, and about just practical issues in the classroom, and so that's really nice for me…. I was a little worried it was going to be the regular members asking questions, the [cluster] leaders answering the questions, but it wasn't like that. We were all asking questions, we were all answering them, and I might have helped ... facilitate a little bit more than others, but there was no big divide between who was talking.

Cycle 1

Even though Clay was an experienced NGPET instructor, he was not satisfied with his grading and assessment procedures. He talked about the concerns he had about assessment during his vcs interview:

There's a lot of frustrations or issues I have with grading, but I would say for me, the big ones were the disconnect between assessment and whatever I want to be happening in the classroom. Like, assessment, not necessarily reflecting student growth… And then, the other part was that assessment necessarily changes student motivation, right? Students are motivated to do well on assessments so that they can get a higher grade, and I want students to be motivated by learning.

During the spring, summer and fall of 2020, Clay learned about alternative grading (AG) strategies from others during cluster meetings. In his vcs interview he talked about the importance of those discussions about alternative grading.

The chance to discuss with other instructors, in this case specifically, about assessment allowed me to just hear other ideas and how people were implementing different things, helped a lot. It's a small group. I can ask questions. We have common experiences with teaching NGPET, so that's this language we speak, and being able to understand the context that we're teaching was important because if it was just me hearing somebody else talk about alternative grading schemes, it's a little harder to put it in the context of my specific environment. So the cluster meetings, having those conversations was certainly a big part.

Cycle 2

During the spring of 2020, with the onset of the Covid-19 Pandemic, faculty in the FOLC switched to teaching their NGPET classes online. This led to a number of challenges, including assessing students’ learning in this environment. Clay joined a small group of FOLC members who were tasked with developing assessment resources for online learning, and he learned more about alternative assessment strategies in his discussions with that group. The group presented a webinar to the entire NGP FOLC community about various assessment strategies that faculty might use in their courses. They followed up the webinar by writing a chapter that was included in a FOLC-publication, Resources for Teaching NGPET Online, that was made available to the whole community. A strategy that particularly interested Clay was proficiency grading.

You start with a list of the learning objectives for the class. What are the skills I want students to have? And then, creating assessments where that is specifically assessed… Seeing that really clicked with me in terms of getting assessment more aligned with what I want happening in my class.

Clay did his own research on alternative grading and found out about another strategy, referred to as ungrading.

… it's basically the idea of eliminating grades. And since that's not actually an option for most of us in the end, essentially students choose their own grades from their own self-assessment.

He also found out about a channel on the Slack communication system that focused on alternative grading and joined that as an additional resource for him to develop his knowledge.

Cycle 3

By early 2021, Clay felt comfortable sharing with his FOLC cluster his own version of alternative grading, incorporating both learning objectives and ungrading strategies. He and two of his colleagues led an entire cluster meeting on 3/5/2021 on alternative grading strategies. Clay began the presentation by asking others to think about the purpose of grading, and then he mentioned that he and his co-presenters were going to talk about grading in ways that might be more equitable and more appropriate.

Clay shared what he intended to implement in his own class later that spring. He talked about what he did in his class during the vcs interview:

What I tried to do is implement a hybrid of the mastery grading where students are being assessed on the specific learning objectives from the course.... I have learning objectives, students know them, exams directly assess them, students get that back, and they can request reassessment at a future date.

The less important part I think is that I do the ungrading. I still like the idea of student self-assessing. And so, I basically have a rubric of what I would do,... so they achieve mastery of 15 out of 20 learning objectives, so that corresponds to this grade. I give the students the rubric … They have to fill out this form and give back to me their self-assessment for the quarter. If their self-assessment is significantly different than what I would have, then I set up a meeting with them to talk it through.

Cycle 4

Clay thought that while his first implementation of his grading scheme was not perfect, it was better in several ways than the traditional grading system he had always used. In his vcs interview he said:

[The grading was now] more ethical for sure because it addresses my concern with students learning … So, giving students multiple opportunities to understand the concepts in the class, I feel much more ethical about that. … [Also] I feel like the alignment between what we're doing in class and what the assessment actually is, is better and more transparent to students. They have learning objectives. They know specifically what they're supposed to be learning. The exams very clearly are addressing each of those, and they know exactly how they're doing on each of those.

He also felt his students were becoming more critical thinkers about the purpose and function of grading than they had previously been.

These are, for the most part, future teachers. And so, giving them a chance to actually critically think about grading, because for the most part, they've never had any other experience other than a standard grading scheme. And so, we spend time talking about not only how the course is graded, but why, and having them actually think about why that might be better or not.

Clay had implemented his alternative grading scheme in both spring 2021 and spring 2022. During that time, he also shared some of his new knowledge with colleagues in his own department.

[In a] typical conversation … my colleague says something along those lines of how they want to change or how they are frustrated with grading in this way or that way. And then, I mentioned that I have made a pretty substantial change in one of my courses, and that I know a little bit about things, and they want to hear more. … At this point, I've probably heard from a large fraction of my department, informally like that, that they want me to actually give a talk to the department about what I've been doing and start a bigger conversation. Because I think if there's multiple people working on something like that, it's a lot easier to make change.

Cycle 5

His personal experience with alternative grading, and his conversations with colleagues in his own department, have led him to re-think both the role of grading in student learning and how he could be an agent for positive change for his own department. In his vcs interview Clay indicated that his experience has validated his initial skepticism about traditional grading and he now sees a more active role for students in the process.

Students aren't just a passive observer in that process, where [now] the students are actively participating in the assessment process and that they have some control over it. Not just the part where they're self-assessing on things, but also that they can see how they've done achieving learning objectives. If they want to improve, they can choose or not to try to improve on them. I think that giving the students more power and things like that has been a positive change too, and I really like it.

He also credits his involvement in the FOLC for supporting a changed view of leadership within his department.

I still don't necessarily view myself as a leader of anything. Of course, I was a cluster leader for a long time [in the FOLC] and just sort of putting myself in a position of leadership like that…. I guess it did transform how I see myself in my department as someone that can step up as a leader within a team to do things.

To visually summarize this case study, Fig. 1 provides a diagrammatic representation of Clay’s value creation story.

Fig. 1

Diagrammatic representation of Clay’s value creation story

Cruz’s case study

Cruz is a tenured faculty member at a Midwest university that is classified by Carnegie as a Master’s College and University. He joined the NGPET FOLC at its inception in 2017 as one of the cluster leads because he was familiar with the curriculum, having already taught it for a few years. He had been using the Physical Science and Everyday Thinking (PSET) curriculum, a predecessor to NGPET, since 2009. He also piloted the new NGPET curriculum for a year prior to joining the community. His reason for joining the community was to expand his collaboration beyond his university to share and learn from others who share similar teaching and learning goals so that he could be more effective as an instructor.

During his post year 1 interview he talked about the importance of the cluster meetings to his value creation process, that the members of the FOLC had a common focus in that they were teaching the same curriculum.

I would say the open exchange of ideas of resources and materials, where everyone was so respectful of what everyone was contributing. … I think that was very much characteristic of the way that the FOLC was set up in being able to share and exchange those ideas and those resources…. With the FOLC [cluster meetings] … it's like you have more opportunities to listen. You have more opportunities to reflect on what's being said…. And so, you're getting multiple perspectives…. communicating with other people, talking about some of the things that worked and didn't work, sharing the resources, and all of those things has been very helpful.

In his vcs interview Cruz talked about the impact his involvement in the FOLC had on his role as Director of Science Education at his university:

As Director of Science Education here at [university], I'm responsible for a number of courses as well as programs. …I felt that my participation in the FOLC had given me insights in terms of how to interact with my colleagues, my faculty, in terms of not only for the course where we implement Next Gen PET, but quite frankly, our other courses.

Cycle 1

In cluster meetings during the first year of the FOLC Cruz talked about how his students were having difficulties writing scientific explanations. The NGPET curriculum introduces explanations in the context of online homework assignments, not as part of in-class activities as had been done in the previous PSET curriculum. In his vcs interview Cruz described some of the difficulties the students were having, specifically connecting the explanation claim to the supporting evidence, and providing cause and effect reasoning. He also talked about how helpful it was for him to have discussions with other faculty about the challenges students were having, and how those discussions led him to think about a research project for the following semester that could focus on scientific explanations.

During our cluster meetings, during our FOLC meetings where we would share with each other resources that we were using, this [student difficulties with writing explanations] came up in those discussions. And I think that actually had a lot to do with identifying a research project that we wanted to work on [the following semester]. And I think in terms of exchanging those ideas [that] was very important. For example, if within the FOLC, within the cluster meetings, … if it did not come up that anybody else was running into the same issue, that having students write scientific explanations was difficult, I'm not so sure that we would be so eager to say, "Okay, let's work on this."

Cycle 2

During Year 2 of the FOLC Cruz joined a small group of faculty members from multiple clusters who wanted to focus on developing tools to help students construct better explanations. He said during the vcs interview:

Here at [University] we had developed a number of tools to help provide students assistance in terms of writing scientific explanations. And what was really special was that there were colleagues of ours within the FOLC who actually did the same thing. And so we were able to pool our resources, and share with each other those resources and strategies that were used.

At the first meeting of the explanations group (9/24/2018) Cruz shared that he didn’t think the NGPET materials provided sufficient support for scientific explanations/writing. He wanted help from the group in coming up with best practices for writing explanations, and in particular, he wanted to come up with some kind of rubric to assess scientific writing. Over the course of the year, the explanations group reviewed and analyzed examples of student writing and produced a set of resources that included storylines, templates, rubrics, prompts, and interventions for promoting student writing. During the group meeting on 2/26/2019, Cruz shared his plan for adapting an idea suggested by one of the other members in the group:

[Member], I pretty much kind of followed your lead a little bit in terms of I took your ideas from your PowerPoint, from your PowerPoint that you shared with your students. And I kind of built upon that in terms of this is a handout that I provide to my students. It identifies the things, like what the energy diagram should have, what the narrative should have, and then there's an example and then there's a rubric.

The discussions Cruz had, both within the explanations group and his own cluster, raised his awareness of the differences between those discussions and the ones he had been having with colleagues at his own institution.

Oftentimes when you're at your own institution, you have a very narrow lens where you focus in only on your students. When we first implemented Next Gen PET here at [university], we would have multiple sections, multiple instructors, where we would meet with each other on a weekly basis. It was similar to the FOLC, but I would probably say that in a local situation like that, oftentimes it's like, okay, I'm aware of what my colleagues are doing, but I'm going to do my own thing. And I think with the FOLC, you have an opportunity to really listen and gain insights on what other people are doing in more detail.

Cycle 3

In spring 2020 the FOLC organized a virtual conference where members could record talks on various topics and make them available to the entire community. This conference provided an opportunity for Cruz to share the resources that the explanations group had developed with colleagues within the FOLC, and he saw that as a culminating experience for him. He also gave a talk at the winter 2021 national AAPT meeting, which was presented virtually.

Cruz also used the resources that he had developed with the group to change what he had been doing in his own NGPET course. In the vcs interview he talked about what he had learned.

I think the one thing that I learned was that it takes … students time to develop well thought out scientific explanations… And I think that regardless of the resources, the ideas, the tools that were used, it's just one of these things that you constantly have to reinforce…. And I think it really just confirmed that there's no shortcuts to this process.

Cycle 4

Cruz’s revisions to his NGPET course had two lasting impacts. First, he found that his students were more successful than previously in applying their scientific writing skills across various curriculum units and modules, and that they improved over the course of the semester. Second, he and his colleagues were able to propose the course to meet the University’s general education requirement, along with a possible assessment and grading rubric related to students constructing and evaluating scientific explanations. Up until now, the course was used exclusively by preservice elementary teachers to meet requirements for their major. By having the course on the list of allowable courses for general education, it would be available to students in any major to enroll, and provide them the opportunity to develop proficiency in constructing and evaluating scientific explanations.

Cycle 5

During the vcs interview Cruz was asked to reflect back on the discussions he was having with colleagues, and to consider how his understanding of success might have changed. He had previously talked about the differences between the conversations he was having with colleagues in the FOLC versus those with colleagues at his own institution. He had implied that he valued the former more than the latter. Now, upon further reflection, he had come to realize that he could value his local conversations more than he had previously thought, and for him that realization was a success.

Locally, there's some distance. Because it's like, okay, this is my section, I kind of know what you're doing, but we get so wrapped up in our daily responsibilities that we just don't sit down, and have that exchange of ideas. Whereas with the FOLC, you have those opportunities, quite frankly, that you don't really take advantage of even with your colleagues locally. And I think in terms of my success, or what I value … it's kind of opened a door. It's like, okay, I should really be focusing on what's going on here locally, so that I can apply some of those same things that I learned within the FOLC. And that's really what I was trying to get at. Being aware of what my colleagues are doing, how they're being successful, how I can apply that into my classroom, versus just focusing on what I'm doing.

To visually summarize this case study, Fig. 2 provides a diagrammatic representation of Cruz's value creation story.

Fig. 2

Diagrammatic representation of Curz’s value creation story

Courtney’s case study

Courtney is a teaching assistant professor at an R1, very high research activity public institution in the South. She teaches both the NGPET course and other introductory physics courses. She joined the NGPET FOLC in 2017 to improve her use of the PET curriculum and has come to appreciate the social connections and opportunities to engage in broader discussions of physics teaching. Prior to 2017 she had taught the studio version of PET since 2005, when she attended an implementation workshop shortly after the PET curriculum had been developed.

When she was interviewed at the very beginning of the FOLC project she talked about what she really liked about the PET curriculum and about teaching PET.

That they [students] really understand that they can do this, without, they can do science without having the teacher walk them through step by step. That they can rely on themselves and not me. And, I think that’s probably the biggest thing that, the most fun thing that I get out of teaching the class actually, is to watch these students, most of them math and science phobic, and to watch them grow to understand that they can actually do this stuff without me holding their hand. And so I think that’s by far the most powerful part of the curriculum. … I’ve really enjoyed teaching PET. I think I’ve learned a lot, actually. And I think teaching PET has helped me become a better teacher in the other classes that I teach. And I’m really looking forward to seeing how PET has evolved, and seeing what else I can learn from it that I can take to other classes.

In her vcs interview, she talked about how her view of teaching has evolved over the years:

I was an undergrad at … a big university where the classes were never small… That's how I learned how to teach because that was how I was taught. Part of that says that when you are now the teacher, what you do is you get in front of the class and you lecture and you give tests. And if people come to you with individual problems, like they're sick and they can't come half the time or whatever, you just say things like, "Well, you should have thought of that before you signed up for the class. Maybe you should drop the class if you can't come all the time," or whatever. …

That is what I was like for the first… eight or ten years that I taught… Then I went to [the workshop in 2005 where] I learned how to teach PET... That already started a change in me thinking about what it means to be a good teacher, and it's not just to stand there and lecture. … I feel like now it's hard to imagine that I was really mean to my students. … I didn't mean to be mean. I was just doing what I thought everybody did, and what so many people still do… I wanted to be able to hold an audience and tell them an exciting story. And "Isn't this neat?" That's very self-centered, right?

In her general vcs she wrote about being a different teacher now that she has been involved in the NGP FOLC and participated in many cluster meeting discussions.

I am sure that I am a different teacher now than I was before. One thing I’ve discovered is a growing desire to change the way I assess my students and determine grades for the class. I can attribute this to FOLC folks who have talked about their efforts in this….

I’ve learned that I can experiment with my classes - trying new things - and that has made my teaching better.

That concern about assessing students led her to write her value creation story.

Cycle 1

Courtney heard about alternative (proficiency) grading (AG) in a webinar organized by the FOLC and which took place in summer 2020. In her vcs she wrote about how those presentations caused her to think about grading, and why she grades, in ways she had not previously thought about. In the following fall she heard some members in her cluster meetings speak extensively about this grading strategy. All these conversations piqued her interest in alternative grading.

Cycle 2

Through the FOLC, Courtney learned about another professional community outside of the NGP FOLC dedicated to discussions of alternative grading. Interested in learning more, she joined that group’s Slack workspace. She recalled this during her vcs interview:

But then it was sometime later,... somebody said something about the Slack, the Mastery Grading Slack… I went and looked and joined, just because I'll join almost anything that looks like it might have potential to help in my teaching.

Later, in a cluster meeting on 5/13/2021, Courtney discussed her involvement in the Alternative Grading Slack and her interest in a virtual conference focused on AG organized by the Slack group. In the cluster meeting she noted “I’m totally looking forward” to the conference and anticipated it helping her implement alternative grading in her teaching.

Cycle 3

Armed with insights and examples both from the FOLC cluster discussions and the AG community, Courtney first incorporated standard-based grading into a non-NGPET conceptual-based physics course in fall 2021. She taught two sections of 200 students each, and she had additional teaching assistants to help with the grading tasks.

I started trying to figure out how I could, being an impatient person, "How can I do this in my very next semester?" …I had to be very clever about how I tried to do that because I didn't want to create a whole bunch of work for myself. …That was a key feature in being able to implement it that first time, was having the manpower [extra TAs] to grade.

As I recall, there were some mentions [in FOLC cluster meetings] that I was doing this, and other people talked about what they were doing for grading.

Cycle 4

She thought her first experience with AG was okay, and she made extensive notes on what changes she might make the next time. At this point she also decided that she wanted to try the strategy in her NGPET course the following semester and looked forward to it.

Cycle 2 again

She turned to the FOLC community to help her. In January 2022 she visited one of the FOLC members who had experience with AG and who provided her with additional resources and encouragement:

We did a lot of talking because she was also, ... doing some alternative grading… I got a lot of ideas there that I thought that I would try, and even some things... I basically stole a whole bunch of stuff from her.

Cycle 3 again

When Courtney next taught the NGPET course, she incorporated alternative grading, revising her implementation based on her earlier experience with the non-NGPET course and the discussions with the FOLC member. In this implementation of AG she decided not to give major tests after each one or two units, but instead to give shorter and more frequent quizzes. She talked about how she explained this strategy to her students.

I tried to explain to them that … “we're going to be doing this other grading scheme where what I'm really interested in is you learning the material.” So, what I did was I gave them a series of quizzes over the learning goals that I had set for the class or the topics that I had decided that they really needed to be able to do. Then I would give them a quiz … that would take up the last 20 minutes or whatever of class … I told them, "What I'm really interested in is that you can learn how to do these things. This is important. There won't be a [major] test. What there will be is a series of quizzes, and if you don't get the quizzes right, then that's fine, but that means that you need to keep working on this, so it's not going to be like a test. … You're going to keep working on it until you get it right or until the semester ends, whichever comes first."

Cycle 4 again

During the interview Courtney talked about how her AG strategy of giving shorter and more frequent quizzes, but no long tests, affected her students. They seemed to be both confused by the strategy, but also relieved. One thing she had learned was that students need to have (major) tests because without them the students may not take their learning seriously.

And they seemed kind of confused by this idea. I was like, "This shouldn't really confuse you because you all want to be teachers and the goal of the teacher is to help people learn. It's not just to give tests and then, oh, well, so sorry, move on." The students really were very confused by that, to the extent that they were like, "Yeah, no tests. Woo," and that feeling of "Woo-hoo, no tests," sort [of] continued on through the semester. That's why … I think the next time I do it, I am going to give tests, but the tests will be short ... [But] I noticed that there were a lot, and probably because there were no [major] tests, there were a lot of people who were not nearly as anxious about being tested … There was a lot less of a focus on, "What do I need to do to get an A?" And more of a focus on, "How do I get this type of question right?" That was helpful.

Cycle 5

Although her implementation of alternative grading is still in the early stages and a work in progress, she already has a new outlook on her role as an educator making learning accessible. As a consequence of her participation in the FOLC in general, and of her experience with alternative grading in particular, Courtney feels as though she is getting closer to what an effective teacher should be, by facilitating learning rather than just being a score keeper. Her teaching has become more student-centered, rather than teacher-focused, and her goals feel more authentic:

[Previously] I'm not even really thinking about the students. I mean, I am. I'm telling them my story, but I want them to listen to my story. My perspective is now changed so that what I want is still, when I'm teaching a big lecture class, of course I still want them to pay attention. It's not like I don't want that, but what I'm [now] more concerned with is what are they getting out of this? What are they learning? Are they learning anything? And if they're not, then what am I doing? How can I do it differently?

In her vcs interview she talked about the role that the FOLC played in helping to change her view of teaching and her implementation of AG strategies in her classes. She felt the cluster meetings, especially those in her first year in the community, were important.

Well, obviously I wouldn't even have known about alternative grading if it hadn't been for the FOLC…. Right from the very beginning, that first year on the FOLC, just having a stable group of people who were so great. .. and just all the good ideas, all the different points of view, all the things, and having every other week to hear people talk about their teaching experiences and their teaching philosophies … I think that was amazing. … Looking back on it, I think that was the first year of (NG)PET, and that was really transformative, I think for me. I think that's really when I started to really shift to what I am now. … I think that's one of the things that makes the FOLC unique, is that we were having conversations every other week with the same group of people…. We were already listening to what everybody in our group was saying.

She mentioned that the transition started when she had attended the workshop in 2005 to prepare her to teach the original version of PET. She learned a lot from that workshop, but in the ensuing years there was not much follow-up with respect to meeting with the PET developers or the people who had attended the workshop with her.

That was great, but there wasn't a lot of follow up. … That's the difference with the FOLC. The FOLC, you're meeting with the same people every other week, and so you start to get ideas from these people and then you're meeting with them again in sort of a reinforcement sort of thing. I think that's something that you don't get with a one-off meeting or even with a drop-in, sort of, "Who's available today? Let's have a meeting," type of meeting. I think it was important that it was the same people meeting regularly and every other week regularly, and not just once a month regularly. That was really reinforcing.

To visually summarize this case study, Fig. 3 provides a diagrammatic representation of Courtney's value creation story.

Fig. 3

Diagrammatic representation of Courtney’s value creation story

Julia’s case study

Julia is a Senior Lecturer at an R2 public institution in the Southwest. She joined the NGP FOLC at its beginning in 2017 in part because she wanted to interact with other faculty teaching the course and in part to learn new ideas that she might be able to incorporate into her own course. At that point she had previously taught the PET curriculum one time and had taught other introductory physics courses for many years. During the first five years of her involvement in the NGP FOLC she was the only instructor at her college who used the NGPET curriculum. Participating in the community during that time has had significant influence on Julia’s instructional practices and outlook on her role as an educator. Her value creation story below is an example of how learning in the community provided value to her.

During the first four years of her involvement in the FOLC Julia taught the NGPET course each year, spread out over two semesters. Her enrollment was mainly Freshman education majors. During that time she had a particular interest in finding strategies to assess students that would encourage them and help promote their learning. She mentioned this during her vcs interview:

Because every semester I had been always kind of figuring it out, always making little tweaks … of how best to grade the students, how to give them best feedback and how to, especially for educators and pre-service teachers, to take away that traditional ABC numerical grading but allow them to redo assignments.

At a cluster meeting on October 7, 2019, she talked about a grading strategy she had used during the second semester of the course:

[In] my second [semester] class, since I have a smaller class of 24, my tests are part multiple choice and then part the short answer narratives…. [For the multiple choice] I have them [use] two different color highlighters, and yellow was their first answer. Then blue was their second answer. I'd give them partial credit if they answered as their second choice [the correct answer]. That way then if they were torn between two, it wouldn't be completely right or wrong…. A lot of them liked that because they were debating between two and it gave them a chance to obviously earn a little bit more points, but it helped them…. [But then some would say] like, "What if I don't have a second choice?" I'm like, "Then that's fine." That did go well.

She also discussed another strategy where students could correct their answers after taking the test and receive partial credit. However, they also had to provide an explanation for why their second answer was correct.

Cycle 1

During the summer of 2020, after the Covid-19 Pandemic had caused classes to be taught online, Julia joined a group of other FOLC members to help write a Resource Guide for Teaching Next Gen PET online. She co-authored a chapter entitled: Online Student Engagement for Next Gen PET and participated in a webinar offered to the whole community. She was also aware of another chapter on that Resource Guide that focused on Assessing NGPET students Online (of which Clay was a co-author). It was in that chapter that Julia would have first become aware that others in the FOLC were implementing alternative assessment strategies, like standards-based grading (SBG) and mastery-based grading. Her interest in SBG was further piqued at a November 6, 2020 cluster meeting, where one of the participants gave an extended presentation on the SBG strategy and how he was using it in his NGPET course.

Julia’s involvement in co-authoring the chapter for the Resources Guide was important to her because as a lecturer in her department she was not involved in research. Therefore she felt her department didn’t always value her contributions. However, she felt her participation in the FOLC did bring her value in providing opportunities to contribute to the community and in recognizing and validating those contributions. At her vcs interview she said:

So, that's I think sometimes hard when you're in those lecturer [positions] in your department you're kind of, you're there but you're not there. Right? … And they like you but, do you have value? So, to an extent, so [being a member of the FOLC] it helped to bring value to yourself really of, I am making a contribution to the bigger [picture]... Right?

In fall 2020 Julia also co-facilitated a group meeting, where the focus was on identifying and addressing student misconceptions.

Cycle 2

Over the following three semesters (S21, F21 and S22) she participated in cluster meetings where others gave extensive presentations on SBG and mastery-based grading (Clay was in the same group for two of those semesters). Julia was impressed by these presentations, and asked questions, especially how she might implement the strategy in her classes where she had about 200 students. In her vcs interview she talked about those presentations:

They were implementing it and they were talking about the good and the bad on it. So it was more honest, more real instead of it being, "Oh this is what ideally works and you should go do it." And then you're like, "Okay."

During the interview Julia talked about how she valued the FOLC meetings during her multi-year experience in the Community. She mentioned the importance of having conversations with others who are teaching the same course at the same time, and how the ongoing and periodic meetings provided the opportunity to reflect on her own teaching.

I think with the FOLC [it] was nice when you are teaching it, and then [with] others who are teaching it. ... Because the conversations would go different paths with our meetings. … What it was nice too is, it gave us time to actually reflect, right? Because we're so caught up in teaching and learning and doing everything else, that it's nice to have time to be like, "Okay, let's kind of reflect what, well, I wonder..." So it was nice to kind of bounce off current situations that were happening while teaching it. Right? And then those who have taught it too and get their feedback too. Right? … It's like that we all have the same goal. Right? That's what's nice. …. It's just having the support of the community and it being dynamic, right?

Cycle 3

The knowledge she gained by listening to and interacting with others during the cluster meetings provided her with confidence and encouragement to implement SBG in her own course, which at the time was taught in the online mode. She commented on how she was going to do this during the vcs interview:

So that was going to change things too because my standards were going to be building on from the previous semester ... The way we taught the course was, they all usually took it in the fall and then the second one they took it in the spring. So they got to build and it was awesome because you got to see them have that growth over the whole year. … So [I] was trying to allow the students, especially to, with the online … to master the assessment, to master it. Right? Instead of it being one and done. … So that's the standards I was trying to implement, and I was trying to get it started with the online sections, so that they had the multiple opportunities to come back. That if they weren't getting it right now, they had time in another couple of sessions [of the course] to get it. And so I was starting to do that more with the assessments …

Julia did also talk about some of the challenges:

Because I don't know if y'all remember I had four classes of 60, and it was just me with no graders. So that too was trying to get, how could I get it a little bit more automated and easier to give feedback to the students when we had so many students.

Cycle 4

During the vcs interview Julia was asked about the satisfaction that implementing standards-based grading brought to her classes.

… it helped for the students to feel less pressure. And it helped for us to feel more of a community of learning with the class of, it didn't have... They got to see more of the connecting, kind of all of the material connecting a little bit better. Because they're starting to see, "Okay, well if I'm not getting it right here, I can [get It later] …

The interviewer followed this up by asking to what extent she felt that her students’ success could be attributed to her using the SBG or whether other aspects of the NGP FOLC played a role. She talked about how the FOLC was important to her because of her isolation at her university. [She was an untenured lecturer and was the only one teaching the NGP course.]

In general, about the FOLC? … Basically, it was just me at [University] fending for myself. And y'all made it awesome to have this community to [talk about] all of the curriculum and how to implement it, and just learning ... It gave me a way to have other people talk to it because I didn't have anybody. Right? Because nobody else was teaching this stuff and had used it. … So it was just nice to have a community where, and I think this is a big aspect too, especially if you're in a not pleasant experience with your current university. It was nice to have someone to talk to that's outside that I didn't have to have, like, "My boss is his [boss]."

As a result of her incorporating SBG, her enjoyment of teaching improved, particularly with larger classes. She has been better able to have discussions with her students, use the inquiry approach, and see students’ “aha” moments in the classroom. As evidence of the positive impacts of her efforts, Julia mentioned during the interview how a department of education faculty member teaching her students in a methods class indicated to her that he has seen an increase in student confidence.

And so they [education department] really liked us focusing on the fact of them [students] building their confidence in science. [The course] it increases their confidence. But not just their confidence, but their knowledge of the nature of science. I think that's a big one. Because that's what I tended to emphasize too, the big theme as well in the course is the nature of science.

More recently her institution set up studio style classrooms (where students can work in groups sitting at round tables), which Julia indicated during her interview had a positive impact on the students’ attitudes about the class.

And I would overhear students, that this was their favorite class, some of them. And that they actually were learning and liked coming.

Cycle 5

The positive experience and feedback on her redesigned NGP course led Julia to reconsider both her role as an instructor and her goals for her students. She now sees her role as being a mentor to her students, encouraging and facilitating their own learning, rather than just someone who teaches them content. Also, whereas her previous goal for her students was to have them learn the essential ideas of science, she also now sees the importance of having her students feel they are part of a community, both while they are learning together during the class and afterwards as they proceed in their trajectory to become teachers.

And it was nice too for them, because it was more of, "Come in, let's work together." And we were all working together, as a class. And you could see the students too that they were getting to know each other and building off each other's, their ideas too. So it felt like, well we're just coming in together to do a class. … [and they were] getting to interact with their classmates too. … because my class was their freshman level class, they didn't know any other education majors. … They're already starting to build a cohort to themselves, right? … So it kind of already started to build a little community there.

To visually summarize this case study, Fig. 4 provides a diagrammatic representation of Julia’s value creation story.

Fig. 4

Diagrammatic representation of Julia’s value creation story

Luli’s case study

Luli is a tenured faculty member at a R2 (Carnegie Classification high research activity) Hispanic-Serving Institution in the southwest. She joined the NGPET FOLC in the first year of the project in fall 2017. Prior to becoming involved in the community Luli had been teaching physical science to preservice teachers (a two-semester course) for eight years and used a traditional textbook. Her students come from the College of Education and most are Latino/Hispanic. In her post year 1 interview, Luli talked about her teaching style:

Usually, I use [an] active learning strategy. Before even I tried this (NGPET) curriculum, I used clicker questions in class. And so, I would help students tease out their misconceptions. … I also use demonstrations. … In order to not get the class feel any boring moments, in my class I often switch from lecture five to ten minutes, and then a question, then stimulation or a video, and come back to the lecture. So, [I] focus on conceptual understanding and the topics that I must cover. It works fine with my students and that's how I ... I'm quite open with student questions, and I always lead them to think about the connections with real-life, real-world applications.

During the vcs interview, Luli said she is recognized in her Department for being a dedicated teacher; she claims one reason for this is that she takes the time and effort to bring demonstrations to her classes to illustrate the physics ideas to her students. Apparently, most other faculty do not make such an effort.

Although she was unable to attend one of the in-person kickoff workshops during summer 2017, she was familiar with the NGPET curriculum because she had attended a workshop at the 2015 AAPT (American Association of Physics Teachers) summer meeting in College Park, MD. Her main reasons for joining the FOLC were to be able to receive support and encouragement from others to counter her perceived isolation at her own university, and to find new ways to stimulate pre-service elementary teachers’ interest in science. The discussions she engaged in while participating in the community for several years have informed her about ways to address student difficulties in learning physics and have reignited her passion for teaching. Her value creation story below is an example of how learning in the community provided value to her.

Cycle 1

At a summer 2017 AAPT meeting one of the NGPET FOLC organizers told her about the community. Since she was already somewhat familiar with the NGPET curriculum and had been thinking about possibly implementing it, this conversation led her to decide to join the community in time for fall 2017.

Cycle 2

Although Luli participated in the FOLC during fall 2017, she did not use the NGPET materials that semester. Instead, she listened to others in her cluster talk about their implementation of the curriculum. At the 11/28/17 cluster meeting Luli said that she was planning to implement at least part of the NGP curriculum in one of her classes during Spring 2018.

At an AAPT meeting in January 2018, a colleague told her about the NGPET magnetism unit, that it had a focus on modeling, students seemed to really like it, and it would be a good unit to try out in her class for the first time. This motivated her to purchase the materials on her own and to try implementing the unit with her students during spring 2018.

Cycle 3

During cluster meetings the co-facilitators often start by having members share something about what is going on in their classes. In spring 2018 Luli talked about implementing the magnetism unit in one of her classes. The goal of the unit is to help students develop a consensus model of magnetism based on evidence they collect through experimentation, either performing experiments themselves or watching videos of experiments. Although most faculty who use the lecture-style materials show the videos, Luli had her students use the materials she had purchased herself. At the 3/1/18 cluster meeting she talked about how she was pleased with the result of her first implementation:

And so for my class, we just finished the magnetic unit, and students were able to reach the consensus model. I felt very happy for them. … the magnetic unit works quite well. Students really like it, and I think they love the thinking and love the discussions, to a point that they can show their reasoning to be when something doesn't work out. … I did provide all the hands-on activities and materials to them. So, I didn't just share the lecture videos. I found that's quite beneficial to them.

During the vcs interview she talked about what she thinks motivates her students, why the NGPET magnetism unit was a good fit for her and her students, and why she felt the need to go out and purchase the materials herself and then go to the trouble to wheel a cart across campus to go teach her class.

I thought because of my teaching experience with this student population, I knew what would promote their interest, to stimulate their interest. My students like hands-on activities in physical science, those hard science classes. In the past I did demonstrations, the number of demonstrations I did is probably ranked very high in my department. So people see me roll my cart with different kinds of things, … Even though it is from my physical science building to cross the campus to the math building, I still roll my carts over just for students to have an authentic experience at this curriculum.

Her successful experience teaching her students with the NGPET magnetism unit motivated Luli to want to expand the use of the curriculum in her classes. At a cluster meeting on 5/1/18 she shared that she planned to read and reflect on new units to implement in the next academic year.

In her post year 1 interview Luli talked about the importance to her of discussions she is involved in during the NGP FOLC cluster meetings:

I think that it really helps that we have the Zoom meeting every other week, and that we get to talk about the student responses, the content that we covered, the technique that we used, our feedback on the curriculum materials. … So it's quite interesting and stimulating for me to improve my teaching. … I find it stimulating each time we talk about it [the NGPET curriculum]. We are sort of a group of pioneers in this curriculum, and so sometimes this kind of communication and feedback are not often happening among local community faculty. And so, when you try to do something innovative, you probably need mutual faculty support. You also get a sense of direction where we go from here. That's what I enjoy the most.

Cycle 4

During her implementation of the NGPET magnetism unit in spring 2018, Luli felt students engaged in conversations that could never happen in traditional lectures. These included students challenging each other’s answers with experimental evidence. She also felt that the students enjoyed their small group discussions, and that students’ understanding of the underlying scientific content was increased. At various cluster meetings she shared her observations about how her students’ classroom talk and reasoning abilities seem to improve, and how pleased both she and her students were:

They talk to each other more. We also ask questions more, whenever they are not sure how to answer certain open-ended questions, but also in the middle of something they didn't know about before. … These [activities] promote deeper thinking for them. [2/1/18]

I feel those skills, reasoning skills, has improved a lot. They were silent in class before, but now they all have something to say. [2/15/18]

We just finished the magnetic unit, and students were able to reach the consensus model. I felt very happy for them. … Students really like it, and I think they love the thinking and love the discussions, to a point that they can show their reasoning to be when something doesn't work out. [3/18/18]

In her post year 1 interview, Luli talked about the evidence she has that students seem more motivated to learn:

You know, nowadays people bring cell phones to [the] classroom. … I noticed that with the [NGPET] curriculum, they look at cell phones much, much less frequently [than] they do without the curriculum. Yeah, I can see the difference. And they often, it stimulated their thinking enough that sometimes at the end of the class when they're all supposed to leave because the class ended, students often will come back and say, "What do you think about that one?" Because still the question is still in their mind because [the] investigation hasn't ended until next class.

Luli teaches at a university where she says that research is more highly valued than teaching. She wrote in her vcs about the importance of the NGPET FOLC in helping her maintain her focus and improve her teaching.

Therefore, investing more time in teaching is an effort one has to make because you are swimming against the tides. With the network that makes faculty members meet about twice a [month], we keep up with each other and the tasks become easier. We get some acknowledgments when we share our teaching experience with each other. It's nice to get to know peers who care about teaching nationwide. It's even better that we share our experience using the same curriculum.

She also wrote about how her discussions with other cluster members kept her engaged in thinking about new ideas and approaches to her teaching.

Exchanging ideas in the network brought values to my practice. As one keeps teaching this course, one could run out of ideas or grow tired of thinking about more new ideas. However, discussing new ideas with colleagues nationwide reignites the passion for my teaching.

Cycle 5

In her post year 1 interview, Luli talked about how her view of teaching has changed because of her experience teaching the NGPET curriculum. She indicates how she now has more confidence that her students can reason things out for themselves without interventions from her.

I felt more confident on students. When they were sitting there listening to me in a regular lecture and demo and so on, I often think that they knew things after I explain to them, or at least I help them correct their misconceptions, and that was my ... I wasn't fully [convinced] that they have the capacity to reason things out on their own with each other's help in a group until last semester. I saw in front of my eyes how much critical thinking they can develop through the process. And I think that's very helpful for their future careers as well, for that reason. I feel I increase [my] confidence in them, and that's the importance of the curriculum.

Her experience in the NGPET FOLC has also led Luli to want to share her students’ success with others in her department.

So, all of those abilities [that] students [are] getting in this course is invaluable, and I've shared it with my department chair and a couple of colleagues. I think that in the spring [when] I teach it again, I'm going to invite one or two of them to visit my class at least once or twice to see how the curriculum has promoted discussions, group discussions, and discovery in the introduction to physical science class.

Luli also gained confidence in sharing her own ideas about teaching and learning during cluster meetings. She now sees herself as contributing to the learning of others in the community, not just within her department.

Usually, what we talk about in the cluster meetings are focused on teaching, focused on our feedback on curriculum, focus on our experiences using the curriculum. … I think after [the] middle of last semester, I feel more comfortable asking questions and don't feel ... I didn't feel awkward. [At first] I felt a little bit uncomfortable asking questions because there were many people in the cluster, and I always had [inaudible] questions as well. … And each group member has something unique to contribute. For example, I work at the Hispanic-Serving Institution. So, how does the curriculum work on Hispanic students? I guess there, it may be interesting to some of our faculty in the community.

To visually summarize this case study, Fig. 5 provides a diagrammatic representation of Luli’s value creation story.

Fig. 5

Diagrammatic representation of Luli’s value creation story

Discussion

In the Findings section we described the case narratives for the five participants of the NGPET FOLC. In this section we first address the two research questions. Then we make connections between value creation and reflective practice. Finally, we discuss some comparative insights that we glean from the five cases. Our intent is to compare the five cases in such a way that the findings have broader applicability and relevance to others.

Research Question 1: what values do members of the NGPET FOLC gain as a result of their participation?

Each of the five case narratives include value creation stories that span across the five cycles of the Wenger et al. (2011) framework. As mentioned in the Theoretical Framework section, each cycle brings a different kind of value, as defined by Wenger et al. The research team identified statements corresponding to the different cycle values from the re-storied value creation stories that the case participants had originally submitted (see Methodology section). Table 2 shows a summary of the cycle values across both the five cycles and the five cases. It is important to acknowledge, however, that these values are based on the specific value creation stories that five case participants decided to write. They could have written different stories, or more than one story, thereby bringing forward different specific values gained from their participation in the NGPET FOLC. So the comparison that follows, as summarized in Table 2, as well as in the FIndings, represents a sample of the kinds of values members might gain through participation in the FOLC.

For Cycle 1, participants were asked to consider a significant event that initiated the value creation process for them. Wenger et al. calls these immediate values. From Table 2 we can see that all but Luli either heard about or discussed some pedagogical strategy: alternative grading standards based grading, or issues involving students’ writing of explanations.The focus on some pedagogical issue as an initiator for the value creation process is not surprising since the FOLC was designed around promoting implementation of a curriculum. For Luli, her value creation process seemed to commence with her hearing about the NGPET FOLC and her decision to join it.

For Cycle 2, participants were asked to consider how their participation changed them in terms of new knowledge gained or new access to resources or people. Wenger et al. refers to these values as potential values, as a next step following Cycle 1 immediate values. As can be seen in Table 2, the case study participants gained new knowledge and new contacts at Cluster meetings (Julia, Julia), content group meetings (Cruz), while working on a set of FOLC resources (Clay), while privately visiting another FOLC member (Courtney), or with non-FOLC groups (Clay, Courtney). Cruz also gained a new perspective on his interactions with colleagues at his own institution.

For Cycle 3, participants were asked to consider what difference participation has made to their teaching of NGPET or their professional life. Wenger refers to these as applied values, because it refers to how the participants made use of the new knowledge, resources or people contacts they had developed in Cycle 2. As can be seen in Table 2, all five of the case participants implemented the new pedagogical strategies in their own NGPET courses. Clay also co-led a cluster meeting focusing on the new strategy, while Cruz gave professional presentations on his new strategy.

For Cycle 4, participants were asked to consider the outcomes, the impact of their implementation of changes to NGPET, or the impact of other knowledge or insights they had previously gained. Wenger et al. calls these realized values. As can be seen in Table 2, all five case participants described positive changes in either their students’ learning, attitudes, thinking, confidence or engagement. Clay became engaged in new pedagogical discussions with local colleagues, Cruz was successful in getting a new general education course approved at his university and Courtney came to a new realization about tests. Julia’s students came to realize the importance of their community, and Luli realized the importance of her own contributions to a community.

For Cycle 5, participants were asked to consider how their participation changed their understanding of what matters, which Wenger et al. refers to as reframing values. One of the criteria for selecting the five cases was that each of the participants achieved values in cycle 5. As can be seen in Table 2, the reframing values cover a wide range of changes. Clay came to consider a new view of grading as well as his role within his own department as a change agent. Cruz came to value his discussions with local colleagues in a more positive and productive way. Courtney, Julia, and Luli all came to reconsider their roles as teachers, while Julia also reconsidered her goals for her students and Luli came to see that her ideas could have a positive impact on a wider community than just her local colleagues.

The reframing values mentioned above are a significant outcome of the five case participants’ involvement in the NGPET FOLC. These new ways of thinking about teaching or learning happened because the FOLC provided both the time and encouragement for participants to learn about new pedagogical approaches, to try them out, and to reflect on their impact.

Research Question 2: what are key ways that members engaged in the NGPET FOLC and changed their professional life?

The case narratives discuss both the Cycle values that the participants gained as a result of their participation and also some of the activities and events that helped them gain those values. In reviewing the five case narratives, the research team identified several themes that touched on the key ways that participants both engaged in the FOLC and developed as professionals. Below we describe these themes, with examples from the case narratives. These themes provide a response to research question 2, and we offer them in a way that could help others involved in FOLCs make connections between our specific curriculum-focused FOLC and other pedagogically focused FOLCs. These themes echo several of the interactions which Kezar, et al. (Kezar 2017) identified as particularly salient in contributing to the process of value creation: exposure to new and innovative ideas, personal interactions in a safe and inclusive space, networking, and mentoring.

Pedagogical discussions and insights

All five of our case participants found FOLC discussions productive and insightful. They all received new ideas and helpful feedback as they engaged with the curriculum (especially Julia and Luli), developed new assessment strategies (Clay, Courtney, and Julia) or introduced new elements to their practice and improved the curriculum (Cruz.) Clay talked about the importance of the discussions in providing him with ideas he could use in his own teaching:

The chance to discuss with other instructors, in this case specifically, about assessment allowed me to just hear other ideas and how people were implementing different things, helped a lot. [vcs interview]

Courtney specifically mentioned both the cluster meetings and the summer webinar hosted by the FOLC as instrumental in providing insights into alternative grading. Several participants mentioned the importance they placed on shared curricular context. Clay, for example, noted how the common language and context helped him think about how he might implement alternative grading:

We have common experiences with teaching NGPET, so that's this language we speak, and being able to understand the context that we're teaching was important because if it was just me hearing somebody else talk about alternative grading schemes, it's a little harder to put it in the context of my specific environment. [vcs interview]

Julia also noted the importance of ongoing, periodic meetings with a consistent set of other faculty who share a similar and concurrent experience. Luli emphasized the importance of the periodic cluster meetings and the pedagogical discussions in stimulating her thinking about teaching the NGPET curriculum:

I think that it really helps that we have the Zoom meeting every other week, and that we get to talk about the student responses, the content that we covered, the technique that we used, our feedback on the curriculum materials. … So it's quite interesting and stimulating for me to improve my teaching. [vcs interview]

Personal interactions, collaboration, networking, and joint initiatives

All five of the participants engaged in networking and collaborations beyond the regular participation in cluster meetings, and these opportunities were significant elements of their stories. Clay collaborated with FOLC members to create resources for online teaching during the COVID-19 pandemic. Cruz joined a FOLC content group to extend his collaborative networks and developed tools for student learning. Courtney joined an alternative grading community, which she learned about through the FOLC, and visited another FOLC member to exchange new teaching strategies. Julia found her experience as a lecturer co-authoring a chapter for a Resource Guide to be particularly impactful, providing her with an academic outlet, recognition, and a sense of validation. She particularly valued the ongoing discussions within the community as a way of combating pedagogical isolation:

Basically, it was just me at [University] fending for myself. And y'all made it awesome to have this community to [talk about] all of the curriculum and how to implement it, and just learning ... It gave me a way to have other people talk to it because I didn't have anybody. Right? Because nobody else [at my University] was teaching this stuff and had used it. [vcs interview]

Luli shared her experiences with her department to potentially inspire her colleagues. Both Luli and Julia noted their interactions with the FOLC were important in countering feelings of isolation. Interestingly, Clay, Cruz and Courney all spoke of engaging in networking outside of the FOLC as important outcomes of earlier interactions within the FOLC.

Changes in practice

Three case participants, Clay, Courtney, and Julia, made changes in how they assessed students, inspired and guided by FOLC discussions. All three adopted alternative grading practices in one or more of their courses, and returned to the FOLC as they refined their use of this approach to assessment. Courtney, in particular, spoke about how her practice has changed because her view of both student learning and her role as teacher has changed:

[Previously] I'm not even really thinking about the students. … My perspective is now changed so that what I want is … I still want them to pay attention. It's not like I don't want that, but what I'm [now] more concerned with is what are they getting out of this? What are they learning? Are they learning anything? And if they're not, then what am I doing? How can I do it differently? [vcs interview]

Cruz changed the way he taught students to write explanations. Luli implemented a specific unit (magnetism) in her course and planned to expand to using additional units. All five refined and revised their practice throughout their participation in the FOLC.

Value creation and reflective practice

One of the initial goals of the NGPET FOLC was promoting reflective practice. Participation in the FOLC prompted several of our study participants to reflect, reconsider, and ultimately reshape some of their instructional philosophies and techniques. Clay, Cruz, and Luli grew in their view of their role within their department or institution. Courtney, Clay, and Julia revised their view of assessment and moved towards a more student-centered view of teaching, in part as a result of reflection prompted by the FOLC's discussions. For example, Julia explicitly mentioned the importance of the opportunity to reflect.

I think with the FOLC [it] was nice when you are teaching it, and then [with] others who are teaching it. ... What it was nice too is, it gave us time to actually reflect, right? Because we're so caught up in teaching and learning and doing everything else, that it's nice to have time to be like, "Okay, let's kind of reflect what, well, I wonder..." [vcs interview]

In the Theoretical Background section we briefly described Rodgers’ (2002) four key features of the process of reflective practice. Reflection emerged as an important theme both in the values our participants identified (RQ1) and the process by which they arrived at these values (RQ2.). In this subsection we will discuss connections between Wenger et al.’s (2011) five cycles of value creation and the process of reflective practice.

Reflection as a progression of experiences leading to deeper understandings and connections

Progression is implicit in the nature of the cycles, starting with cycle 1 which initiates the progression and experiences and cycle 2 in which the community member develops resources and knowledge to deepen understanding. Deeper change in practice occurs in cycle 3, with results in cycle 4, and of course arriving at the ultimate goal of reflective practice, revising one’s understanding of success and adopting new goals and strategies, in cycle 5. In choosing narratives that span all five cycles, we have also identified examples of reflection as a progression of experiences leading to deeper understanding and connections.

Reflection as a rigorous and systematic examination of practice

The cycles offer multiple opportunities for intentional and systematic examination of practice. In cycle 2 participants develop knowledge, skills and connections that they can develop and use in transforming their practice. We have shown in the narratives that all of our participants engaged in critical examination and further revision to move from cycle 3, the application of their new knowledge, to cycle 4, seeing improvement and success, and to reframing in cycle 5. Cruz describes this in his interview:

With the FOLC [cluster meetings] … it's like you have more opportunities to listen. You have more opportunities to reflect on what's being said…. And so, you're getting multiple perspectives…. communicating with other people, talking about some of the things that worked and didn't work, sharing the resources, and all of those things has been very helpful. [vcs interview]

Reflection through interaction with others in a community

Community interaction is at the heart of both Wenger’s framework and the NGPET FOLC. Wenger explicitly presents community activities and interactions, networking, connections, and feedback from the community as essential to value creation. These were also highlighted by our study participants as key elements in their narratives and sources of inspiration for insights. All five participants noted how they were able to get relevant feedback at the right time from the community of trusted colleagues:

Julia: It was nice to kind of bounce off current situations that were happening while teaching it. Right? And then those who have taught it too and get their feedback too… It's like that we all have the same goal.

Clay: So, the depth of the FOLC conversations tend to be greater, but other than that, I think it was a lot of the same ... I mean, we all have the same issues, right? We talked about a lot of the same things

Cruz: What was really special was that there were colleagues of ours within the FOLC who actually did the same thing. And so we were able to pool our resources, and share with each other those resources and strategies that were used

Courtney: One of the things that makes the FOLC unique, is that we were having conversations every other week with the same group of people

Luli: We are sort of a group of pioneers in this curriculum, and so sometimes this kind of communication and feedback are not often happening among local community faculty

Reflection as growth of the Individual and of Others

Each cycle represents some aspect of growth, whether developing potential value or changing practice. Cycle 5, however, represents reframing and reevaluating success, in close correspondence to transformative change as the ultimate goal of reflective practice. All of our study participants noted new outlooks on teaching and several also grew in their professional roles. Clay and Courtney explicitly spoke of these changes in their thinking during their interviews:

Clay: I guess it did transform how I see myself in my department as someone that can step up as a leader within a team to do things.

Courtney: My perspective is now changed… what I'm [now] more concerned with is what are they getting out of this? What are they learning? Are they learning anything? And if they're not, then what am I doing? How can I do it differently?

Comparative insights

The five case narratives provide detailed stories of how the participants gained value through their participation in the NGPET FOLC. Although the value creation stories told in these narratives are different, the responses to the first research question reveal many similarities in the kinds of values gained. We now look across the five case narratives to draw insights that focus on: participants’ roles within the community and prior curriculum experience; challenges of assessment; the number of local colleagues teaching the NGPET curriculum (and with whom they can engage in pedagogical discussions); leadership opportunities provided by the FOLC; and features of the FOLC that support participants value creation.

Participant roles in the community and past curriculum experience

Three of the case participants (Clay, Cruz and Courtney) came into the FOLC having extensive experience with the curriculum pedagogy, while two of them (Julia and Luli) had little or no experience. Nevertheless, through their participation over the years they all gained significant value. Clay, Cruz and Courtney joined the NGPET FOLC as cluster leads. They all had many years of previous experience teaching either PET or PSET, and Clay and Cruz had also piloted early versions of the NGPET curriculum. Their task as cluster leads was to both facilitate cluster meetings, encouraging all participants to speak and share ideas, and to bring their experience to bear in helping to address implementation issues as they arose from others who were new to the curriculum. So, it was not clear at the beginning of the FOLC that they would also gain value through participation. However, all three spoke about doing so during their vcs interviews. Clay talked about the frustrations he had with grading during the first two years of teaching NGPET. During cluster meetings in the third year of the FOLC, he learned about alternative grading strategies, did additional research, and successfully implemented new assessment strategies in his own classes. Cruz was concerned that the new NGPET curriculum was not providing students with sufficient scaffolding for writing scientific explanations. He joined a small group during the second year of the FOLC to develop supplemental resources to scaffold students' writing. When he implemented these resources in his own class, he found them to be very helpful. Courtney first became very interested in alternative grading strategies when she heard about them during a FOLC webinar in summer 2020. Even though she was an experienced instructor of the curriculum, the webinar caused her to think deeply about her own grading strategies. She then learned more about alternative grading and implemented it in her classes, with some success.

Julia joined the FOLC having taught the PET curriculum just once, and Luli joined having attended only a PET curriculum workshop a few years prior. As such, like many other FOLC participants who had little or no prior experience with the curriculum, during their first year of participation both Julia and Luli were mostly concerned with management and logistical issues of implementation (Anderson, 1997). In time they each gained significant value through their continued participation. However, because Luli initially focused on implementing just a single unit (magnetism) rather than the whole curriculum, and because she had previously used active learning strategies in her teaching, she was able to achieve success even after one semester, noticing significant changes in her students’ classroom interactions and attitudes. During her first few years implementing NGPET Julia tried to make small changes in how she assessed students and talked about them at FOLC cluster meetings, but she wasn’t entirely satisfied, especially since she was teaching classes with large enrollments. She learned about alternative grading strategies by reading the assessment chapter in the FOLC Resource Guide for Teaching Next Gen PET Online, and in cluster meeting discussions over the following couple of years. She implemented some of the strategies in her own classes and found that the approach helped students feel less pressure about assessment, see more connections between ideas, and feel they were part of a community (of learners).

Challenges of assessment

Three of the case study participants (Clay, Courtney and Julia) cited alternative grading or assessment strategies as bringing value to them through participation in the FOLC. However, they implemented the new strategies only after they had participated for a few years in the Community. Adopting new assessment strategies is challenging, especially in interactive and collaborative courses, which may explain why it took time for participants to learn about the novel strategies and implement them. These challenges are consistent with results from a survey that participants filled out early in their experience in the FOLC (data source II in Table 1). The survey asked about preparedness to do various things related to the NGPET curriculum, including structuring their courses, managing logistics and equipment, teaching effectively, and assessing student learning. Participants reported the lowest levels of preparedness to assess student learning, though this increased over time (Priceet al., 2021). Not only is assessment challenging, it also deeply connects to structural incentives, the nature of grades, and the mismatch between traditional grading and a more collaborative curriculum like NGPET. Clay's comment from his vcs interview captures this well:

…the big [frustrations I have about grading] were the disconnect between assessment and whatever I want to be happening in the classroom. Like, assessment, not necessarily reflecting student growth… And then, the other part was that assessment necessarily changes student motivation, right? Students are motivated to do well on assessments so that they can get a higher grade, and I want students to be motivated by learning.

This frustration was shared by Courtney and Julia, and presumably by other members of the FOLC who participated in discussions about alternative grading practices. Working collaboratively through the FOLC, faculty were able to explore and address their approaches to assessment and grading.

Sole NGPET instructor at local institution versus more than one

Courtney, Julia and Luli were the only faculty members at their institutions who were teaching the NGPET curriculum during most of the time they were participating in the FOLC. As such, their conversations about teaching NGPET were mostly constrained to the cluster meetings and other FOLC-sponsored events. Courtney felt the cluster meetings were very important to her:

Right from the very beginning, that first year on the FOLC, just having a stable group of people who were so great. .. and just all the good ideas, all the different points of view, all the things, and having every other week to hear people talk about their teaching experiences and their teaching philosophies … I think that was amazing. [vcs interview]

Julia valued her participation in cluster meetings to share ideas about teaching, especially the NGPET course, since she didn’t have that opportunity as a lone implementer at her college:

So it was nice to kind of bounce off current situations that were happening while teaching it. Right? And then those who have taught it too and get their feedback too. Right? … It's like that we all have the same goal. Right? That's what's nice. …. It's just having the support of the community and it being dynamic, right? [vcs interview]

Luli talked about the importance of the cluster meeting discussions to motivate her to continue seeking to try new strategies in her teaching:

I find it stimulating each time we talk about it [the NGPET curriculum]. We are sort of a group of pioneers in this curriculum, and so sometimes this kind of communication and feedback are not often happening among local community faculty. And so, when you try to do something innovative, you probably need mutual faculty support. You also get a sense of direction where we go from here. That's what I enjoy the most. [vcs interview]

Unlike Courtney, Julia and Luli, Clay and Cruz were each part of a team of several local NGPET implementers. During the post year 1 interview Clay talked about how his discussions with local colleagues about the curriculum might differ from those he had during his FOLC cluster meetings.

There's a lot of similarities. I think the FOLC conversations can get a lot deeper a lot of the time, because the local conversations, a lot of time it's thinking about pressing issues of what activity are we doing next week, and what's the exam going to look like, and just normal team-teaching kind of stuff. And, we don't always have the time to think, and talk about deeper pedagogical issues. Sometimes we do, but not always, whereas FOLC, we talk about all of that stuff all the time. [post-Year 1 interview]

Cruz similarly valued his discussions with his FOLC colleagues, both during cluster meetings and during his discussions with other members of the special content group on helping students write scientific explanations. It is interesting to note that, while Clay and Cruz didn't talk about the value of the FOLC for addressing isolation, they did talk about the FOLC as providing deeper discussions than their local group. They also got resources (alternative grading and materials for scientific explanation writing) from the FOLC (not locally) to help address dissatisfaction with their practice.

Leadership opportunities

Clay, Cruz and Courtney were brought into the NGPET FOLC as cluster leads, so they had leadership responsibilities from the start. They were responsible for facilitating meetings with their co-cluster leads, helping to ensure that participants had the opportunity to share ideas and raise teaching issues of concern. Clay talked about his role in his vcs interview:

I still don't necessarily view myself as a leader of anything. Of course, I was a cluster leader for a long time [in the FOLC] and just sort of putting myself in a position of leadership like that…. I guess it did transform how I see myself in my department as someone that can step up as a leader within a team to do things. [vcs interview]

As he became more experienced with alternative grading strategies, he was willing to share his experiences with local colleagues, but generally only if they came to him with questions. At some point he was asked by colleagues to present a department seminar about alternative grading. Cruz gained a different perspective on leadership as a consequence of his involvement in the FOLC. He was already a leader within his own department, as director of science education, and led periodic meetings with a group of his colleagues who all taught the same curriculum (initially PSET and then NGPET). Over time he found those meetings to be less valuable to his own development as an instructor. However, by facilitating FOLC meetings and observing others do so, he came to view the value of his local meetings differently.

I felt that my participation in the FOLC had given me insights in terms of how to interact with my colleagues, my faculty, in terms of not only for the course where we implement Next Gen PET, but quite frankly, our other courses. [vcs interview]

The situation for Julia and Luli was quite different. They joined the FOLC as regular participants with little or no prior experience teaching the curriculum pedagogy. Over the first three years they participated in cluster meetings, gained experience with the curriculum, and implemented new strategies in their own courses. In fall 2020 the FOLC leadership team decided that because just about everyone in the FOLC had several years of experience teaching the NGPET curriculum, it was no longer necessary to have meetings facilitated by just the original cluster leads. Instead, once cluster membership was decided (mainly on scheduling logistics) all members of each cluster were offered the opportunity to co-facilitate at least one meeting during the semester, focusing on topics of interest to them. Julia and Luli were thus able to take on the responsibility of co-leading one meeting within their respective clusters.

Features of the FOLC that support participants value creation

During the vcs interviews the case participants were asked to comment on features or activities of the NGPET FOLC that seemed to support their value creation process. All five of them emphasized the value of the periodic and ongoing cluster meetings where participants could share ideas and issues of concern and have those issues addressed. Although Clay, Cruz and Courtney were initially chosen as cluster leads and therefore had primary responsibility for leading the cluster meeting discussions, they also found value as participants in those meetings, sometimes learning new teaching or assessment strategies from others.

Clay, Cruz, Courtney and Julia talked about the importance of everyone in the FOLC teaching the same NGPET curriculum. This shared focus enabled people to talk about their problems of practice using a common language, so they could construct solutions for the challenges they encountered such as developing alternative grading systems or strategies to help students write better scientific explanations. Clay and Courtney also mentioned that because the FOLC had continued over multiple years, they had the opportunity to get to know other participants very well, promoting trust, which made it much easier for them to share their own implementation challenges. We note that the narratives of Cruz and Julia, as well, continue over several years and reflect transformative value after long-term participation in the FOLC.

Cruz and Courtney also emphasized the importance of the content groups and the virtual conference that were set up during the second and third year of the FOLC. These activities were a primary vehicle within the FOLC where faculty (including the case study participants) could troubleshoot teaching challenges in the context of classroom practice, share information and resources, and provide a source of encouragement and moral support. Price et al. (2021) referred to these as mediating processes which have an important role in helping faculty increase their knowledge and use of effective pedagogical techniques.

Four of the case participants talked about the availability of specific resources as being important to them: Cruz talked about the use of FOLC constructed Google Docs, and Clay, Courtney and Julia referred to the FOLC publication, Resources for Teaching NextGen PET Online, as being very helpful during the Covid 19 Pandemic because of the shift from in person to online instruction. Finally, Luli talked about the social norms evident at meetings where people respect others’ ideas, making it easier for her and other participants to raise problematic issues and expect constructive suggestions for how to address those issues.

Limitations

We note several limitations to this study. It primarily relies on the perspectives and self-reported values of the FOLC members, without examining other data sources such as classroom observations, products produced, or student performance. This results in a lack of indicators for some cycles of value creation. The small sample size in this multiple case study limits the findings' generalizability beyond the observation that experienced members can potentially derive value from their participation in a FOLC. Furthermore, the values identified in the study are limited to those related to the specific stories offered by the participants. We analyzed just one story provided by each participant; therefore, the list of values obtained may represent only a partial reflection of the benefits derived from FOLC participation. A more comprehensive discussion of value creation in this context can be found in Goldberg et al. (2023).

Conclusion

Participation in the NGPET FOLC was a transformative experience for the faculty members included in this multiple case study. All five participants in this study were able to address major pedagogical concerns and described being supported by participation in the FOLC. Other values noted by participants included developing new leadership strategies and finding community, ameliorating their previous sense of professional isolation. A recurrent, and unexpected, benefit was the way in which the FOLC helped participants find a variety of ways to address the challenge of assessment.

Several features of the FOLC emerged as crucial factors in value creation. Intentional structural elements, such as regular meetings with a trusted community and shared focus, facilitated consistent engagement. Leadership and networking opportunities also played a significant role. All five participants leaned on the FOLC in supporting changes in their practice, from initial ideas to implementation and then feedback on revisions. Importantly, our study participants identified the extended participation and continuing feedback from multiple perspectives as drivers of reflection and growth. The extended participation also seemed central to the success in developing new assessment strategies.

Our participants included both faculty with extensive previous experience with the curriculum, and who served as leaders within the FOLC from its inception, and faculty who were new to the curriculum and joined as community members. At the start of their participation, the experienced faculty were primarily occupied by their leadership responsibilities, and the less experienced faculty focused on implementing the curriculum. One participant (Luli) gained a new perspective on her role as a teacher, although she did so in a somewhat more narrow context than the other participants. By the third year of participation in the FOLC, the remaining four participants also moved beyond these initial concerns and all five reported that they gained important, transformative values from their participation regardless of their role. The discussions within the FOLC also evolved, moving from being primarily focused on logistics and implementation in the first year to deeper pedagogical issues in later years. This suggests that while short term participation in a FOLC or other faculty community may help address more prescribed issues, longer term participation supports a more emergent approach to faculty development.

In conclusion, the NGPET FOLC effectively supported members in gaining significant value. The five case studies provided rich and comprehensive narratives of value creation within the five cycles of Wenger et al.'s framework. By participating in the FOLC, these individuals were able to address major pedagogical concerns, leading to the development and implementation of innovative educational interventions. The FOLC's bi-weekly meetings and extended duration, virtual conferences, and resources were all instrumental in supporting value creation for the participants. The NGPET FOLC stands as a testament to the transformative power of collaborative learning communities for experienced members.