Exploring the influence of collaborative data-based decision making among teachers in professional learning communities on teaching practice

Abstract

Educational policies such as the No Child Left Behind Act in 2001 and Every Student Succeeds Act in 2015 have emphasized the need for hard evidence such as standardized test data to make educational decisions. Therefore, teachers have been expected to use student assessment data to make informed curricular decisions and adjustments in teaching practice. This has led school districts to turn to data-based decision-making (DBDM) to create an effective process of student assessment data analysis, and reflection during professional learning communities (PLCs). The study explored teachers’ collaboration on the DBDM when teachers were analyzing student assessment data from common, district, and state assessments, as well as its influence on their instructional decisions. Participating teachers’ department and PLC meetings were recorded, and individual teachers’ classrooms were also observed. Teacher interview data and their lesson plans were also included and analyzed qualitatively. Results revealed that the purpose of collaboration determined types of data used, modes of analysis, and the impact on their teaching. Furthermore, teachers used varied assessment data to plan and adjust their instruction. Their motivation to enhance their teaching practices was also increased by exchanging ideas and insights rooted in the data.

Introduction

Collaboration amongst educators can lead to the sharing of innovative teacher practice (Woodland et al., 2013). Further, collaborating over student data through a systemic data-based decision-making process can enable more informed teacher practice (Crone et al., 2016). Increasingly, educators are being asked to use data to make curricular decisions. Educators’ use of student assessment data to make instructional decisions has become increasingly important due to the copious amount of student assessment data available to teachers (Fox, 2010; Mandinach, 2012). Earl (2013) outlined three approaches to classroom assessment: assessment of learning, assessment for learning, and assessment as learning. Within these approaches, assessment as learning emphasizes teachers utilizing assessment to modify their practices and enhance student learning. This underscores the role of teachers in leveraging assessment data as valuable resources to tailor their lessons to meet the individual needs of their students. Nonetheless, research has found that often teachers struggle with using data to drive instruction (Fox, 2010; Schildkamp et al., 2014). To combat this, initiatives such as professional learning communities (PLCs) have been implemented to create a student-centred focus to curricular planning (DuFour et al., 2016). DuFour (2004) stated that collaboration amongst teachers should focus on student learning and outcomes. Collaborative groups of teachers in PLCs should operate around three crucial questions: What do we want each student to learn? How will we know when each student has learned it? How will we respond when a student experiences difficulty in learning? (DuFour, 2004). To answer these questions, teachers need to be focused on student outcomes using frequent data collection to drive next steps in instruction. Systematic processes such as data-based decision-making (DBDM) have been implemented to streamline the data analysis process (Gummer & Mandinach, 2015; van der Scheer & Visscher, 2016; van der Scheer et al., 2017; van Geel et al., 2017). However, little research has been done on the specific instructional decision making that occurs amongst teachers who use student assessment data, and the subsequent employment of those decisions which may lead to increased student achievement (Ebbeler et al., 2017; Farley-Ripple & Buttram, 2013; Gummer & Mandinach, 2015).

The purpose of this study is to explore the impact of teacher collaboration on the DBDM with student assessment data from common, district, and state assessments, as well as its influence on teacher practice. Since the specific decision making that results from DBDM, as well as the effect of collaborative data analysis has been under-researched (Supovitz & Morrison, 2015; van der Scheer & Visscher, 2016; van der Scheer et al., 2017; van Geel et al., 2017), this study has the potential to fill a gap in this area of education practitioner research. Slavin et al. (2013) stated that variances in teacher performance with DBDM exist, so their collaborative efforts and instructional decisions made during the DBDM process may be translated differently into individual teacher practice. As such, we set out to examine teachers’ collaboration types and their effects on their instructional decision making and teaching practice. The research questions that guided this study include (1) To what extent do teachers analyze student assessment data and apply their interpretations to plan their instruction?, (2) What types of teacher collaborations occur concerning student assessment data?, and (3) To what extent do teachers implement instructional interventions to improve students’ learning outcomes?

Literature review

Data-based decision-making (DBDM) in professional development

DBDM can be defined as an iterative, cyclical process (Crone et al., 2016) which involves data literate teachers gathering relevant student data that is then translated into useful information (Fox, 2010) to critically self-reflect on teaching and make curricular decisions (Gummer & Mandinach, 2015; van der Scheer & Visscher, 2016; van der Scheer et al., 2017; van Geel et al., 2017). Teacher collaboration over student assessment data can lead to more informed teacher practice and the sharing of effective strategies, and the shedding of ineffective strategies (City et al., 2009). In education, the term data can be defined as any measurement of student information which gives information for educators to use to direct learning or school processes (Wayman & Jimerson, 2014).

Collaboration in PLCs

Collaboration amongst educators is essential for school improvement and increasing student learning (Woodland et al., 2013). Collaboration amongst individuals includes working cooperatively as well as learning together, and that this collaborative work amongst educators can take on many forms including groups of people with differing roles (e.g., educator, administrator, professional, researcher etc.) working together (Robutti et al., 2016). Ronfeldt et al. (2015) found that teachers who participated on instructional teams found their collaborative practice helpful in developing their instructional strategies. Schools saw high student achievement in both math and science when teachers worked on highly collaborative instructional teams which focused on continuous improvement in practice (Ronfeldt et al., 2015).

Student assessment data analysis and teaching practice

A key principle of a PLC is its results orientation which involves teachers actively, and collaboratively, looking for evidence from frequent formative assessment which shows that their instruction is leading to student success (DuFour et al., 2016). For a successful PLC implementation, structured collaboration time and leadership support such as securing teachers’ time to participate in PLC activities play a critical role in successful PLC implementation (Farley-Ripple & Buttram, 2013; Supovitz & Morrison, 2015). Marsh et al. (2015) found that teachers who were engaged in a PLC often used student assessment data to provide students extra support or to reteach a topic, but rarely used it to change the delivery of the content without the support of administration or instructional coaches to encourage change. Whereas changes to delivery were often the result of dialogue about data in conjunction with mode of instruction. More changes to instructional delivery have been noted to occur, when schools and leaders used PLCs to engage in conversations about instructional responses to the data gathered (Marsh et al., 2015). This finding implies that conversations about data during PLCs can be powerful opportunities for teachers to reflect on their practice and effect change (Benade, 2018). Mann and Walsh (2013) supported groups of teachers reflecting together to co-construct knowledge to improve teaching practices and increase student achievement.

Methods

This study was designed to investigate the impact of effective collaboration on the DBDM that occurs amongst teachers when analyzing student assessment data from various assessments, as well as its influence on teacher practice. As the specific decision making that results from DBDM, as well as the effect of collaborative data analysis have been under-researched (Supovitz & Morrison, 2015; van der Scheer & Visscher, 2016; van der Scheer et al., 2017; van Geel et al., 2017), this study potentially fills some gaps in this area of education practitioner research.

An exploratory case study approach in a natural setting was used in this study. This method is appropriate because it investigates an intervention which did not initially have clear outcomes (Baxter & Jack, 2008; Creswell, 2013; Hesse-Biber, 2017; Yin, 2014) in an understudied area (Datnow & Hubbard, 2015; Gummer & Mandinach, 2015) while determining the effects of collaboration on DBDM and teaching practice (Baxter & Jack, 2008; Creswell, 2013; Hesse-Biber, 2017).

Research participants and context

Five teachers at a junior high school (7th through 8th grade students) volunteered to participate in the study. All participating teachers were teaching science and coming with varied levels of experience in education and participation within PLCs. Table 1 presents participating teachers’ backgrounds in education and teaching experiences. The junior high school (NJH) that this study took place in was in the southern U.S. and had an enrollment of approximately 1050 students. NJH was a seventh through eighth grade school having opened in August 2016. This school was chosen for a few reasons. At this school, the students tended to perform well, with most students making the minimum grade considered passing by the state. However, the percentages of students performing at higher levels was not where administration thought that it should be. There were also gaps in performance between all student averages and those in subpopulations, particularly those students in special education or labelled as Emergent Bilingual. This problem occurred year after year on the state tests. To help combat these deficiencies, steps were taken to help teachers better utilise student data to make curricular decisions, the district had decided to begin training campus staff on what a PLC is, key features of PLCs, and how to begin implementing PLCs at each school. This presented a unique situation for the study, as we could see how teachers integrated new learning into their existing practices, or used it to change their practices, to increase student achievement. At this particular school, the teachers look at data as a group. First, the teachers reviewed data presented silently and annotated anything they noticed about that data. These noticings must be directly gleaned from the data and cannot include conjecture. After a few minutes of independently making note of the data, the group shared out. The point in this practice was to make sure teachers stay grounded in what the data was telling them rather than relying on assumptions and anecdotal evidence. After some time discussing what they saw in the data, the teachers then generated questions that stemed from what they noticed. Again, these should be questions that focused on the data and what it was presenting. These questions were then used to reflect on teaching practice to identify areas of strength and weakness.

Table 1 Study participant information

Additionally, the state this study took placed in assesses eighth grade science students each year. This assessment covered standards taught in sixth and seventh grade in addition to the ones taught in eighth grade. The format of this assessment meant that the curricular decision making from grades six to eight must be streamlined and effective to ensure success of eighth grade students on the state examination.

Data sources and collection

The primary data collection included documents such as meeting observations and interviews with participating teachers. This study was approved by the IRB board at the cooperating university and all teacher participation was voluntary. To protect teacher identity, all teacher names are pseudonyms. Data collection began in the spring of the 2019–2020 school year, paused during the COVID-19 shut down, then recommenced in the spring of 2021. Initial data collection started with a focus group interview (FGI) consisting of all participating teachers. Focus group interviews are more conversational and allow for the gathering of new ideas in exploratory research such as this study (Hesse-Biber, 2017). Teachers’ lesson plans, lesson schedule calendars, and PLC planning documents were collected. PLC documents included protocols for data analysis and grading period calendars. Observations occurred in two types of meetings. Department meetings occurred at least once per week whereas PLCs occurred once per grading period (every six weeks). Weekly department meetings were led by the department head for grade level. During these meetings recent assessment data was discussed with the intent of determining if instructional adjustments needed to be made in the teaching that followed. The PLCs lasted a full day and served to plan out the next grading period’s instruction. During this time the teachers would look at historical state assessment data to determine effective and ineffective teaching from previous years, the amount of time needed to be allocated to teach each standard for that grading period, as well as where assessment needed to be implemented to gauge learning. Observations that took place during department meetings and PLCs allowed the author(s) to see the amount of collaboration that occurred during DBDM to see how the teachers reacted to and analysed the student assessment data. The first author also conducted observations in individual teacher’s classrooms. In total, twelve classroom observations (395 min) were conducted. Semi-structured focus group and individual teacher interviews were also conducted to provide further insight into observations. These interviews allowed teachers to provide their own context to the study both individually and as a group. Interviews were recorded using a video recording feature on a laptop computer. Verbatim transcripts were typed up for each interview. Transcripts with interpretations were sent to each participant for review to build trustworthiness. The questions for the interviews generated from the literature and the observations. The first author kept the interview questions open-ended and general, to preserve the perspective of the interviewee and maintain focus on the phenomena under investigation (Creswell, 2013). There were two focus group interviews and several individual semi-structured interviews (15 interviews, 219 min total). The first focus group interview was conducted with all participants and occurred at the beginning of the study. The second focus group interview also had all participants and occurred at the end of the study.

After meeting observations, the first author conducted a round of semi-structured interviews with each participating teacher. These questions were generated from previous observation and document analysis. Example interview questions were “What role does collaboration play when analyzing student data?”, “How often do you analyze data collaboratively?”, and “How does that impact your teaching?” The study ended with a final round of individual interviews and one full department FGI.

Data analysis

This study involved the collection of data from both focus group (2 times) and one-on-one (15 times) interviews, classroom (12 times) and meeting (8 times) observations, and PLC documents such as planning calendars. Due to the exploratory nature of this study, we did not want to approach data analysis with preconceived codes. Rather, an emergent coding analysis was employed. Each piece of data went through several iterations. Interview transcripts and observation notes were coded line by line emergently when behaviour was as collegial collaboration over data, as well as implementation of interventions to meet all students’ needs (Charmaz, 2014; Creswell, 2013). Specifically, the initial iteration of data analysis involved applying interpretations to the data. These interpretations of transcripts or protocols were then sent to individual participants for verification.

In the next iteration, these initial codes were then given focused codes, with similar lines of text receiving the same code. Then those codes were condensed into emergent themes. Code descriptions can be found in Tables 2, 3 and 4 in the following section.

Table 2 Grounded collaborative analysis with codes, descriptors, and examples

Table 3 Collaborative structure with codes, descriptions, and examples

Table 4 Interventions to increase student achievement with codes, descriptors, and examples

To look for patterns across individual participants, as well as grade levels, we looked at the codes applied to individual interviews and classroom observations. The number of each code under each theme was counted for each participant and then converted to a percentage based on the number of codes received by other participants. These percentages were used to see how the themes varied across participants as well as by grade level (7th versus 8th grade). We note that we observed and collected data from an existing PLC. The data collected was then analyzed iteratively, with input from an external qualitative research expert, to identify emerging themes and patterns. This process allowed us to gain insights into how teachers naturally collaborate and utilize data. Also, by consulting and checking our emerging themes and data analysis process with a qualitative research expert who holds a PhD in qualitative research methodology provides evidence to support validity of our analysis methods (Maxwell, 2013).

Trustworthiness

This study involved multiple source points of data such as participant interviews, persistent field observations, gathering of relevant documents, reflexive journaling for triangulation (Creswell, 2013; Hesse-Biber, 2017; Lincoln & Guba, 1985); and data analysis protocols and analysis from prolonged engagement (Hesse-Biber, 2017). This technique allowed researchers to provide support for, or shed light, on a theme or perspective from the investigation, involving looking for common themes across all data that is collected (Creswell, 2013). This study took place over several weeks at a junior high school, during the spring of the 2019–2020 school year, and then throughout the 2020–2021 school year. Also, data interpretation results were shared with participants to seek their verification. The first author held the positions of campus testing coordinator and academic facilitator at the same school as the participating teachers. While this arrangement could potentially introduce bias, it also provided the researcher with the advantage of operating within a data-rich environment and having pre-established rapport with the participants. The researcher had previously served as a teacher at the junior high school for three years. Building trust and confidence among study participants is a critical aspect of research (Creswell, 2013), and this is often achieved through prolonged engagement within the research field (Lincoln & Guba, 1985; Lincoln & Lincoln, 2005). While inter-rater-reliability (IRR) is commonly used as a measure of agreement in qualitative research analysis, researchers emphasized that validity in qualitative research is not solely dependent on IRR (Maxwell, 2013) but can be achieved through other methods such as triangulation, peer debriefing, and rich description (Guba, 1981). For example, we performed the debriefing function with other professionals who were college professors with PhD in education.

Results

Below, we presented emergent themes about how teachers analysed student assessment data.

Theme 1: grounded collaborative analysis

Grounded collaborative data analysis refers to codes which involve the group of teachers making instructional decisions based on data. These codes also indicate that multiple views of the data such as overall performance and standards-based performance were used to gain a comprehensive view of student performance.

Participating teachers stated that they often looked at student data, whether from an informal assessment during teaching or a formal assessment such as a quiz or test, immediately on their own. After the teacher’s initial independent analysis, they met either a grade level (Martha, Pam, and Tina; or Katie and Hermione) or a whole department (all five teachers, plus nonparticipants) to analyse the data and make instructional adjustments. To allow for multiple perspectives and vertical alignment from 7th to 8th grade science in the data analysis, the group met as a department (both grades) for major assessments such as district assessments or benchmarks. Table 2 lists the codes related to this theme, along with descriptors, and examples from the interviews and observations.

Code 1. Reflective, comparative data analysis

The code reflective, comparative data analysis referred to instances where teachers used student assessment data in a comparative manner. Teachers would compare the overall performance of their students with those of other teachers in the group. They also compared performance on individually assessed standards with each other. This type of analysis revealed individual teacher’s strengths and weaknesses which led to the sharing of strategies that may help to increase overall achievement across the grade level. During PLCs, we observed that teachers used historical test data from previous years’ district and state assessments coupled with recent years’ school calendars and lesson plans to reflect on how the data resulted from their teaching. In interviews, all teachers expressed that this form of data analysis had an impact on their teaching practice, as it helped them to see effective strategies they should keep and share and ineffective strategies they should shed. For example, Tina had previously discussed how she had worked at a previous school as the only eighth grade science teacher. In those instances, she mentioned the difficulty in working on a team of one as she had no means of comparison to determine the effectiveness of her teaching, “I’ve also been at another school where I was just the only eighth grade science teacher so collaborating with a little bit more difficult. It’s a lot better when you have people in house to collaborate with and bounce ideas off.” This may be why she relied so heavily on comparative analysis when reflecting on her teaching. Pam, Tina, and Martha all taught on the same team. Pam (22%) and Tina (30%) both had higher percentages of codes regarding reflective, comparative data analysis than Martha (13%). This could be because Martha, while a veteran teacher, is new to the campus. Whereas Pam and Tina, stated they were trained on this type of analysis by a former department head and the principal.

According to Hermione (Table 2, example 2), it is important to compare data so that teachers can share strengths and improve on weaknesses. She also stated in an interview, “we are not looking at just the way that the data is laid out we’re not looking individually at how each teacher did… [It’s] more like what do we need to do to bring that standard’s percentage up (Hermione).” Being reflective led to the shedding of ineffective teaching strategies, as well as the sharing or curating of new ones, all with the goal of increasing student achievement. Pam indicated that comparing across classrooms helped her to plan how she can reteach with her students or approach a topic from a new perspective (Tables 2, example 1). She was reflective in that if her data revealed that there was weakness in her teaching, but strength in another teacher’s practice, then that was a resource she needed to tap into to refine her practice. The same could be said about Hermione (Tables 2, example 2 and 3). Having those types of conversations with colleagues indicated there was trust where teachers could be vulnerable with each other in their analysis and reflection, and a level of willingness to adapt or change their individual practice as needed.

Code 2. Multi-faceted data view

The theme of multi-faceted data view involves the common test or quiz data used during the collaborative DBDM process and the viewpoint from which it is analysed. Data would be looked at in terms of how students performed overall, but this did not fully reveal why they saw those results. Therefore, teachers would dive deeper into the data by looking at how students performed on individual standards and assessment items as well. While the teachers in this department looked at overall performance by grade level, teacher, and class period, they also looked at the data through multiple views such as breakdown by standard and item analysis. One of the most common data views referenced in interviews and seen during observations was the item analysis, where teachers broke down individual questions. During this analysis they looked at the question format, as well as how the item corresponded to the standard, to make sure that it was correctly addressing the content that was taught. Katie (Table 2, example 4) demonstrated that teachers also used item analysis to not only see how many students were getting the question correct, but also to identify possible misconceptions that may exist.

During PLC days, the teachers used a standards-based analysis to see how students performed on state standards overall, focusing on the standards to be taught in the upcoming grading period. This approach guided their planning regarding the order in which state standards were to be taught, how long should be spent on each standard within the grading period, as well as what teaching strategies appeared to be effective. Katie (Table 2, example 4) indicated that using a standards-based analysis showed the teachers which teaching strategies were effective or ineffective. If major problems were identified, the teachers reached out to each other for help.

Tina (Table 2, example 5) demonstrated that teachers also analysed data by class level (advanced versus on-level). This indicated that teachers would assess if their differentiation for ability level was effective. Students in advanced classes were given more rigorous assessments to demonstrate higher-level thinking while measuring the same standards as assessments in on-level classes. As Tina mentioned, by taking their data analysis from an overall view to a class level view, teachers could see if their interventions and extensions were effective.

Interviews with Pam had the highest percentage of codes regarding multi-faceted data views (53%). She often referenced using multiple views of data including overall performance, as well as item state standards based analysis, when looking at student assessment data. Pam claimed that the variety in data views, coupled with multiple perspectives from teachers, was powerful in revealing what the data was saying to her and the group. Katie’s percentage (20%) was interesting because her reflective comparative analysis percentage was lower than other participants. She mentioned that she used data more for analysing state standards as well as identifying student misconceptions. This indicated that she looked at data from multiple views, but from the interviews, did not reflect on her teaching as much. Neither Hermione nor Martha mentioned looking at data from multiple views. Hermione did discuss how comparing data with each other was important, but never discussed looking at the data from multiple views such as by overall performance, item or standard.

Code 3. Collaborative DBDM

Collaborative DBDM refers to any reference teachers made to collaborating and making decisions over student data. The science teachers indicated that collaborative DBDM was important to their success with students as it helped them to see the data from multiple perspectives.

Pam had a high percentage of collaborative DBDM codes from her interviews (29%). She placed a lot of emphasis on looking at data together and discussed how integral this process was to get the full story about the student assessment data. This was also indicated in Pam’s example (Table 2, example 6). In other words, even if the teachers compared their data to another teacher’s, it was still only their personal perspective in the interpretation as they were not in each other’s classrooms. They did not always see what each other was doing that might be effective or ineffective for students. When teachers collaborated in DBDM, they were able to see the data from multiple points of view, thus illuminating evidence not seen on their own as highlighted by Hermione’s example below.

This year we’ve done the ‘I notice, I wonder’, which kind of takes the pressure off like ‘you did a bad job’ in looking at the data overall. [We’re] just trying to help [with] student growth, so I feel like when we collaborate that way, we’re talking about things (Hermione).

As such, the data protocol kept teachers focused on the story the data was telling. Therefore, the teachers remained focused on what was essential to increase student success and were able to be more reflective in their teaching. This was further demonstrated by Martha. In an interview with her, she said that teachers should collaborate over data, but this effort was only helpful if teachers used that data to improve their practice in the immediate and future teaching.

Code 4. Application of historical data

All the teachers mentioned historical data (11% of codes) use in their analysis and planning. However, Martha had the most codes regarding application of historical data from her interviews. She stated that looking at the standards analysis to break them down was helpful, but also that looking at previous assessment items also informed her teaching approach. Examples 7, 8, and 9 from the table demonstrated how the department used historical data while planning collaboratively (Table 2). From the historical data they determined standards that were weak overall and reflected on how that content was taught in the past. They used this data to keep effective teaching, shed ineffective teaching, and target the acquisition of new strategies which may increase achievement:

I really liked this year when we looked at the questions and the data from last year’s district tests and how successful students were with the upcoming six weeks unit. So, seeing that and collaborating, and discussing what it was that was hindering success for that standard (Pam).

By looking at the historical data through multiple data views, teachers were able to hone in on strengths and weaknesses in their own teaching. This allowed them to plan more intentionally in the current year to increase student achievement on those types of items as well as on the standards to be taught.

Theme 2: types of collaboration

The group of participants for this study were highly collaborative, meeting often in either formally planned or informal, random meetings. From the data, two types of collaboration emerged within the department: Proactive and Reactive Collaboration. Throughout the study the type of collaborative structure demonstrated an impact on how data analysis occurred as well as teaching practice in the classroom. Table 3 shows codes related to the theme of collaborative structure along with code descriptors and examples from the interviews and observations.

Code 1. Proactive collaboration

Proactive collaboration and planning involved teachers using data to plan for upcoming instruction. This type of collaboration occurred during the PLCs each grading period and in weekly department meetings in preparation for the teaching that would occur the following week. This collaboration was intentional and meant to create the structure of teaching as well as share strategies which have been proven effective by historical data. These meetings used historical data analysis to identify which may best serve the current group of students. The teachers worked proactively to develop and fine tune their lesson plans based on trends and patterns in historical data, so that all teachers would be instructionally aligned for upcoming units.

During the PLC days provided each grading period (once per six weeks), departments met for the entire day. They paced out their calendar based on the district scope and sequence, as well as by using historical assessment data from previous state and district assessments to identify trends and patterns in the data which may reveal standards that needed more emphasis than others. This was a time that teachers dissected the standards to truly grasp what was required of them in teaching. Martha (14% of proactive collaboration codes) demonstrated that having protected time to plan together allowed teachers to break down the standards needing to be taught, as well as more effectively plan for upcoming units (Table 3, example 2). She explained how having the time to break down standards and look at previously used test items was important to help her get into the mindset of teaching new content at the secondary level.

Additionally, with the use of data, the teaching can be planned in a more targeted manner to best meet the needs of students based on historical trends in the data, as well as what current students may need instructionally. As Katie mentioned (Table 3, example 1), this proactive collaboration helped the teachers focus on the students, as well as be aligned in their teaching. Proactive collaboration can help to streamline teaching, keeping it more efficient so that the focus can be on student learning and progress.

In the interviews, Katie was the only teacher who mentioned that this type of collaboration was perceived as the most impactful. However, Hermione has the most proactive collaboration codes from her interviews (38%). This could be attributed to her role as a department head as the review of historical data is important for guiding the planning process during the PLC planning days, as well as weekly meetings set to prepare for the next week of teaching.

Code 2. Reactive collaboration

Reactive collaboration often occurred after a quiz or test was given during weekly department meetings. This can be seen in Table 3 (examples 3, 4, and 5). The teachers analysed data starting with overall performance and then looking at a correlation between the teaching done and how students responded to each item on the assessment (Table 3, example 3). During this collaborative analysis, the teachers were comparative, looking for individual strengths and weaknesses, as well as strengths and weaknesses across the grade level. This comparative analysis was used to determine what was working well, and what instructional adjustments needed to be made the next week, such as reteaching of content or increased small group instruction.

[From the data] if one classroom has a greater success than another classroom, you can address small changes that need to be made. (Pam).

The example from Pam above demonstrated that during reactive collaboration, the teachers were comparing each other’s data.

Theme 3: teachers’ instructional decision changes

The goal of their meetings over data was to find solutions and interventions which lead to increased student achievement. Table 4 shows the codes related to this theme along with descriptions and examples for each.

Code 1. Small group intervention based on data

In classrooms, the way students were grouped and the activity they do would vary, depending on their need(s). Sometimes students would be grouped together based on similar abilities as demonstrated by historical data. Pam and Martha both stated that they used this method when implementing station type activities. They also both had the highest percentages of small group codes with 31% of the codes from both classroom observations and interviews. As stated by Pam (Table 4, example 2), small group instruction was often the most effective intervention tool for her. Pam grouped students in different ways for small group instruction so that sometimes she was the main teacher, or so that peer tutoring could occur. This demonstrated a data-based approach to small group instruction where students were pulled homogeneously based on similar capabilities, or varying capability in heterogeneous grouping.

Code 2. Intentional interventions

From the data we could see that intentional intervention was important to this group of teachers. Intentional interventions to increase student achievement codes were applied to text segments which indicated that the intervention in place was planned for proactively or reactively, both based on data. This means that the intervention was intentionally planned with the intent to help increase student achievement. For interventions to work, as Hermione explained (Table 4, example 4) that they must be intentional.

Teachers in this group frequently planned time for intervention. Pam and Tina had the highest percentages of this code with 52% and 30% respectively. The state in which this study took place measures students at the end of each year in certain grades and subjects. The performance of students on these assessments was used as an accountability measure for the school’s rating (A, B, C, D, or F). As the teachers at this school wanted their school to be rated high, they worked hard to intervene instructionally for students, so that all students could perform at a high level. This might explain why the eighth grade teachers had higher percentages for this code than seventh grade teachers as they have the pressure of state testing at the end of the year. They may have felt the need to intervene intentionally and often to increase student achievement. However, as demonstrated by Pam (Table 4, example 3) we could also see that having planned the time for the intervention was important to them as they wanted students to be successful. As Hermione noted (Table 4, example 4) the time for intervention needed to be intentionally planned, and what happened in the classroom also needs to be planned. This demonstrated that the teachers allotted time for intervention to address student needs based on data, as well as plans for those students who have mastered the content and did not need intervention.

Code 3. Instructional adjustments based on data

The code of instructional adjustment based on data was used when teachers made adaptive, instructional changes in the moment based on data and/or observations. According to Pam (31% of codes), this data was invaluable and used frequently in her classroom:

Informal data, to me…. I think it is used the most. I think unintentionally almost because you are using it in class, daily like feedback (Pam).

For Pam, instructional adjustments in the classroom observations were seen in the upfront support that she provided to students in areas where they are known to struggle. In one classroom observation, she frontloaded her teaching by reviewing formula triangles and how to use them before having the students work independently in their station activities. She then pulled small groups with targeted strategies to help students where they struggle.

This code was also applied when teachers talked about changing their instruction from one year to the next based on recent data. Martha (33% of codes) demonstrated this in the example below:

I think it helps as a teacher to really know how in the past it has been asked, and the question form, as well as what images, the pictures… to help bring that to the classroom (Martha).

Interventions and instructional adjustments can vary based on student need. Frequently the teachers grouped students who showed similar learning outcomes based on data. However, according to Pam, sometimes whole group interventions were used based on the data (Table 4, example 7). Martha reflected a similar point of view below.

It might have just been the way I taught it, that maybe I did not explain it well enough too. I really want to look at where [the students] are weak at. I look at the percentages and I think about how it was taught. Then I can base my review off that. (Martha).

As she analysed assessment data, Martha looked at the standards where students were weak. She recognized that there was something missing in her teaching, and therefore was sure to reteach that content to make it clearer, as well as spiral it back into the review.

Code 4. Structured and planned assessment to gauge student progress

Teachers constantly assessed students to ensure that they were progressing in their learning and increasing in achievement. Especially those that taught the eighth grade, they had significantly more of this code in their observations and interviews, Pam (33%), Martha (24%) and Tina (24%), than the two seventh grade teachers, Katie (12%) and Hermione (6%). From the calendars created during PLCs, it can be seen that these teachers did plan to have assessments during their units of teaching to gather evidence of student progress, and they mentioned the use of this data in the interviews. Structured and planned assessments were standards based and intentionally planned assessments. Teachers checked for understanding in multiple ways. Sometimes these checks are informal such as, “with your fingers show me on a scale of 1 to 5 your level of understanding.” However, an intentionally planned assessment is one that is written to truly gauge what students do and do not know at critical points of instruction so that adjustments can be made if necessary. The purpose of these assessments was to gauge student progress throughout the content being learned. Teachers used these assessments to reflect on their teaching, and either extend or intervene based on student need (Table 4, example 8).

Purposeful assessments seemed important to these teachers. While some assessments like online interactive games were not relied upon as much, for the most part the formative and summative assessment data were used by the teachers. Especially, formative assessments were often embedded within the learning as demonstrated by Hermione in the example below:

Having [the students] do some exit tickets, I was able to go through those exit tickets and pull out the kids and put them into three groups – They got it, they were kind of getting it, or they were completely lost (Hermione).

Hermione also mentioned later in the interview that if you didn’t use the data from a formative assessment, then it was a waste of time for teachers and students.

Assessment continued even after the quiz or test. Teachers would provide a data based reteaching, either whole or small group, and then assess the students again to ensure that their learning progressed:

At the end of small group, I’ll have some sort of activity for them to do on their own. I’m just monitoring them to see where they go with it… to see if they’ve gotten a better grasp of the concept (Pam).

This demonstrated that just relying on the reteach was not enough. To know if students have corrected or extended their learning appropriately, the teachers must continue to assess. The practice of continued assessment indicated that teachers were constantly monitoring students to make sure they were fully prepared for the state assessment and summative assessments (final exams) given at the end of the year.

Discussion

The participating teachers valued their collaboration in PLCs and were appreciative for the protected time to meet. They reflected on their practice while using this data and were willing to adapt their practice accordingly to achieve the goal of increasing student performance. In this study, we found a relationship amongst types of collaboration and the approach to data analysis, and teachers’ instructional decisions. Specifically, during proactive collaboration, the teachers often used data from prior years’ district and state assessments with the purpose of reflecting on previous teaching practices which were found to be effective or ineffective based on the data. They looked at each assessment item and interpreted the results to find student misconceptions. Based on the item analysis, teachers sought new ways of teaching to remedy unsuccessful teaching for the new school year. This supports a claim by Kaufman et al. (2014) that DBDM should involve teachers analysing data while subsequently reflecting on the teaching strategies that led to those results, and then make curricular adjustments as needed. By looking at individual item performance and reflecting on teaching, teachers would determine if their teaching was matching the level and method at which the content was assessed. If not, they changed their teaching methods to best meet student needs. Additionally, multiple data views, such as looking at overall student performance, performance based on individual standards, and item analysis, allowed the participating teachers to see patterns in the data, as well as find evidence regarding effective and ineffective teaching strategies. All teachers referenced the importance of this type of collaboration when planning their lessons.

Reactive collaboration refers to collaboration which occurred within a day or two after a common or district assessment data was given. During these data meetings, the teachers mainly used current assessment data from district level or common assessments and focused on students’ overall performance along with item level analysis. This is different from proactive collaboration, which occurred before teaching and used historical data to make instructional adjustments. During reactive collaboration, teachers would compare their current students’ data with the intent of identifying areas of strength and weakness in teaching amongst themselves. From proactive collaboration, we found that they sought new methods of teaching by discussing which teaching strategies were effective or ineffective from historical assessment data. Teachers also discussed next steps regarding whether to reteach the concept(s) immediately or move on with teaching but help low performing students in upcoming learning via warm ups, small group activities, or additional homework. According to Benade (2018), necessary skills for reflective practice include the ability to reflect critically on their own practice, as well as a willingness to exchange feedback, which requires both honesty and courage, key characteristics needed for dense, effective networks of people (Bourdieu, 1986; Coleman, 1988; Häuberer, 2011). The teachers in this study aligned with this research as they were reflexive and used these comparative analyses to look at ways in which each teacher can improve so that overall student achievement was increased. As Doubet (2012) found, collaborative teamwork when analysing formative assessment data not only helped to identify what students were struggling with, but also helped teachers hone their assessment crafting skills.

Although Bocala and Boudett (2015) claimed that teachers often do not know how to use data to make curricular adjustments, when looking at the collaborative efforts of this group of teachers, it appeared that they not only frequently analysed a variety of data, but also used them to adjust their teaching practice from year to year, and week to week. This indicated that looking at multiple forms of data in a reflective and comparative manner was important to these teachers. Thus, the results from the current study showed the positive effect of the DBDM process on teachers’ practices in analysing and interpreting data and reflections on their practices.

In summary, the decisions being made depended on the purpose of the collaboration and what data was being analysed at that moment. For example, while looking at historical data over physics standards, the teachers found that items asking students to interpret tables and graphs were the lowest performing ones. Therefore, when planning for that grading period, they were intentional in adding more opportunities for students to construct their own tables and graphs from data collected during a lab investigation or curated, as well as analyse data and graphs with the content further demonstrating how historical data can be used to make changes from year to year.

The eighth-grade science team showed frequent instructional adjustments from week to week during the chemistry unit. Meeting frequently while using data to reflect on teaching and identify areas of student need allowed more opportunity for them to adjust their teaching to increase student achievement. Conversely, the seventh-grade team used historical data to plan but did not reflect as much on current data or observations to make instructional adjustments. The results support that teachers need to have the fortitude to analyse data, reflect on teaching, and most importantly willing and able to make instructional adjustments in the classroom based on reactive collaborative data analysis.

Collaboration over data is essential for school improvement (Robutti et al., 2016; Woodland et al., 2013). By collaborating over data, each teacher brings in their own individual experiences and influences to help to strengthen all students and increase their achievement. The level of trust and vulnerability required to participate in this type of data analysis demands strong connections between educators (Häuberer, 2011). To build the strength of their network, teachers need to be willing to share effective strategies, as well as curate or create new strategies when present ones are deemed ineffective based on student assessment data (Pil & Leana, 2009).

Implications and limitations

This study was conducted on a small group of teachers in one school and one content area department. Further research could be done in other content areas and different school settings. Research in this manner would be beneficial to administration, district leaders, and educational leaders as it would help to demonstrate characteristics of highly collaborative teams and how that collaboration enables teachers to disaggregate data during DBDM and make necessary instructional adjustments with the goal to increase student achievement. It could also help administrators and district leadership see where support is needed to better aid teachers in the PLC process as well as data analysation with the intent of making curricular adjustments. Support could be found through training, the addition of instructional coaches to schools, and district personnel whose expertise lies in these areas. Additionally, teachers would gather as an ad-hoc basis to discuss their teaching and students’ learning. Further research could focus specifically on those ad-hoc meetings between teachers and how they impacted their teaching practice.

Conclusion

Teachers’ collaboration can take on many forms. Depending on the form it takes, collaboration has varying impact on the type of data analysis done amongst teachers, as well as how they share strategies and adjust their teaching practice. To make a structure for upcoming teaching, we claim that proactive collaboration is necessary where planning lessons is based on trends and patterns. Effective planning can be done in this manner. However, teachers also need to be sure to gauge student progress throughout learning by intentionally planning assessments which test student comprehension and capability with the content. They need to act reactively to those assessments by collaborating over the data to make instructional adjustments based on student need. This is perhaps the most important type of collaboration as teachers can pre-plan all they want, but if they do not address student needs in a timely manner, students may not progress with the content. For this type of collaboration to occur, we recommend that teachers have protected time to collaborate (Farley-Ripple & Buttram, 2013). Here, supportive leadership could play a critical role in ensuring dedicated collaboration time for teachers (Farley-Ripple & Buttram, 2013; Supovitz & Morrison, 2015).