Student interaction discourse moves: characterizing and visualizing student discourse patterns

Abstract

Student-centered instruction allows students to take ownership over their learning in the classroom. However, these settings do not always promote productive engagement. Using discourse analysis, student engagement can be analyzed based on how they are interacting with each other while completing in-class group activities. Previous analyses of student engagement in science settings have used methods that do not capture the intricacies of student group interactions such as the flow of conversation and nature of student utterances outside of argumentation or reasoning. However, these features are important to accurately assess student engagement. This study proposes a tiered analytical framework and visualization scheme for analyzing group discussion patterns that allow for a detailed analysis of student discourse moves while discussing scientific topics. This framework allows a researcher to see the flow of an entire conversation within a single schematic. The Student Interaction Discourse Moves framework can be used to extend studies using discourse analysis to determine how student groups work through problems.

Introduction

Classroom discourse

There has been a push for educational institutions to engage in student-centered and collaborative instructional approaches in Science, Technology, Engineering, and Mathematics (STEM) classrooms designed to help students construct their own understanding and take ownership of their learning (Freeman et al., 2014; National Research Council, 2015; Olson & Riordan, 2012; Theobald et al., 2020). These approaches have shown improvements in students’ exam scores, overall course performance, pass rates, and retention rates in STEM courses (Freeman et al., 2014; Loes et al., 2017; Nussbaum, 2008; Wilson & Varma-Nelson, 2016). When having an emphasis on viewing science as “a way of thinking rather than a body of knowledge”, studies have also shown positive impacts on students’ reasoning, problem-solving, communication, and critical-thinking skills (Erduran & Jiménez-Aleixandre, 2007; Talanquer & Pollard, 2010; Wilson & Varma-Nelson, 2016).

Aligned with Vygotsky’s collaborative learning theory, the benefits of learning environments that include group work are centered on the role of classroom discourse and positive peer interactions (Vygotsky, 1978). Discourse is defined as “any stretch of language (spoken, written, signed) which hangs together to make sense to some community of people who use that language”(Gee, 2015). In other words, discourse is the language we use to interact with people in a community. In the context of science, discourse involves asking questions, constructing arguments, and building explanations that can be shared with the scientific community to help understand new phenomena (Lemke, 1990; Osborne, 2010).

Specifically in chemistry education, discourse analysis serves as a qualitative tool to explore the role of classroom talk in students’ development of content knowledge, scientific reasoning, construction of arguments, and the establishment of a shared understanding of phenomena (Cole et al., 2014; Driver et al., 2000; Gee & Green, 1998; Kulatunga et al., 2014; Mercer, 2010; Moon et al., 2016, 2017; Repice et al., 2016; Talanquer, 2014; Young & Talanquer, 2013). Research has shown that discourse moves such as engaging in reasoning, expanding students’ thinking, developing good listening skills, and building accountability as they work towards a common understanding promote the development of the desired knowledge and skills. (Criswell, 2012; Michaels & O’Connor, 2015; Michaels et al., 2008; Towns, 1998). Specifically, students have been encouraged to request clarification or justification, challenge the ideas of their peers, and share what they understood from a previous utterance (Criswell, 2012; Michaels & O’Connor, 2015; Moon et al., 2017). By examining group dialogues of chemistry students regarding solution chemistry, Warfa et al., (2014, 2018) determined that sociochemical norms, which constitute the discursive norms of the classroom that regulate what counts as a chemical justification and explanation, altered how students reasoned about the cause(s) of chemical phenomena. It was noted that students’ statements such as confirmatory and clarifying questions, agreeing or rejecting with reasoning, seeking group consensus, and acknowledging each other’s ideas allowed students to reach a collective understanding of chemistry concepts (Warfa et al., 2014, 2018). Similarly in discourse research, Repice et al. (2016), explored the use of language in peer-led small groups in a first-year general chemistry course as students worked through different problem types. Their study demonstrated that students engaged in regulative talk (communicative interactions that involve everyone present in the space) in order to manage group discussion and dynamics between peers and used instructional talk focused on problem solving to develop their understanding of chemistry concepts (Repice et al., 2016). In addition to cognitive benefits, other studies on collaborative learning and scientific classroom discourse have shown positive impacts at a social and personal level (Eren-Sisman et al., 2018; Hamnett et al., 2018; Nichols, 1996). If members of a group recognize themselves as active participants working towards a shared goal, they can experience higher levels of group trust, perceived consensus, and gain satisfaction that improves group discourse (Harney et al., 2017; Repice et al., 2016; Wells & Arauz, 2006).

Despite these numerous possible benefits, studies have shown that collaborative learning environments do not necessarily lead to productive group engagement, which we define as students engaging with each other’s ideas in a manner that expands students’ understanding of relevant concepts and skills and that support collaboration and inclusion of all group members. Some studies have reported interactions such as antagonistic dynamics between peers, unequal member participation, lack of thoughtful arguments, and focusing on task completion with minimal discussion occurring more often in low-performing groups than in high-performing groups (Barron, 2003; Chan, 2001; Ryu & Sandoval, 2015). Even when these conflicts are not identified, other studies have found that groups only spend a small amount of their discussion time engaging with the content (Summers & Volet, 2010). They even suggest that students tend to avoid high-level content processing interactions such as elaborating, interpreting, and reasoning, which are usually students’ best opportunity to learn from each other (Summers & Volet, 2010). Poor group dynamics are a concern as classroom dialogue reflects the social dynamics and norms that exist within a group that shapes the quality of group learning (Gee & Green, 1998; Mercer, 2004). These findings align with views that students need to be taught how to engage in effective group discourse, and suggests the need for additional investigation of the role of social interactions on collaborative learning in STEM (Osborne, 2010). The work presented in this paper addresses these concerns by demonstrating a method for analyzing both the cognitive and social aspects of student discourse within STEM classrooms. Specifically, this paper builds on the work of existing analytical frameworks within scientific discourse analysis and illustrates how to visualize student discourse for further qualitative analysis. The paper begins by reviewing established methods for analyzing and visualizing student discourse and discussing why a new framework is needed to further analyze student discourse in STEM classrooms. This is followed by a description of the design of the Student Interaction Discourse Moves (SIDM) framework and a method for visualizing the applied framework for analysis. The paper will conclude with implications for research using various lenses to show how this framework can be applied to different research contexts.

Theoretical framework

Vygotsky uncovered that as a group of individuals collectively processes information through clarifying and exchanging ideas with each other, each member constructs their own understanding (Vygotsky, 1978). Thus, each member plays a critical role in contributing to the discussion by responding to previous utterances to advance the discussion to a collective understanding. When students are willing to listen and critically examine alternative perspectives, and when members of the group generate new ideas that extend beyond their initial understanding, then “progressive discourse” can be achieved (Bereiter, 1994). Such engagement in collaborative discourse also provides students the opportunity to build reasoning skills or reconstruct incorrect reasoning (Wegerif et al., 1999). Thus, under a social constructivist perspective, learning extends beyond the accumulation of knowledge and describes the progress of speech, thinking, and behavior that allows individuals to effectively participate in social and intellectual activities (Wells, 1999). These processes and behaviors also define productive engagement when students are working together to complete a task. In support of better understanding of how we can promote student learning in a collaborative environment, we sought a methodological approach to investigate the flow as well as the patterns of student interactions through the lens of social discourse.

Review of discourse analysis frameworks

The analytical frameworks developed by Sinclair and Coulthard (1975), Coulthard (1992), Kaartinen and Kumpulainen (2002), Sampson et al. (2011) & Sampson and Clark (2011) each provide insights into student discourse, although none address all the interactions that are part of productive discourse. This led us to develop the SIDM framework to analyze student discourse, taking direction and inspiration from these earlier analytical frameworks.

Sinclair and Coulthard developed and refined a model called the Initiation-Response-Follow-up model (IRF) that investigated the social structure of discourse in a classroom, primarily between students and instructors at the secondary education level (Atkins, 2001; Coulthard, 1992; Sinclair & Coulthard, 1975). This was some of the first patterned discourse work done and was found to be useful for mapping focused, one-on-one conversations that highlighted the utility of being able to code student discourse across a whole conversation. However, we found this framework lacked the ability to be used for free-flowing small group conversations with more than two members and did not readily capture who or what idea a student was responding to in a conversation. This framework also left out the nonverbal interactions that are sometimes present in a conversation; both critiques have also been mentioned by others in more recent literature (Atkins, 2001).

Kaartinen and Kumpulainen (2002) incorporated Sinclair and Coulthard’s (1975) and Coulthard’s (1992) ideas in a framework they created to analyze how college students in a collaborative inquiry setting constructed their understanding of the definition of dissolving. Part of their analytical framework examined the nature of students’ participation in the manner of initiating, continuing, extending, referring back, agreeing or disagreeing, replying, commenting, and concluding while discussing scientific ideas. The other layers of their framework (logical process, nature of explanation, and cognitive strategies) aimed to characterize students’ explanation-building processes (Kaartinen & Kumpulainen, 2002). This study inspired the idea of designing a tiered framework, as well as the types of codes one could use to characterize an utterance. However, the framework itself was focused on the reasoning patterns of student conversations and did not address types of student discourse outside of cognition. Our framework includes cognition codes like reasoning and presenting claims, but we wished to also characterize the broader social interactions such as motivation, information sharing, and questioning.

Sampson, Grooms, and Walker.(2011) developed a framework to capture how students engaged in scientific argumentation using Argument-Driven Inquiry (ADI), when working in small lab groups. This framework was then expanded outside the lab environment to the classroom environment, where Sampson and Clark (2011) examined students’ argumentation patterns by characterizing high school science students’ responses (agree, reject, discuss, and ignore) and the functions of their utterances (information seeking, exposition, opposition, and co-construction). For a closer examination of the nature of students’ oppositions, they described what criteria students used to evaluate the claims presented in their group. In this way, the researchers were able to differentiate between high and low-performing groups (Sampson & Clark, 2011). From this framework, we found inspiration for writing more specific codes, such as the types of questions students ask each other. However, similar to the Kaartinen and Kumpulainen (2002) framework, this framework was narrowly focused on students’ abilities to construct arguments under the ADI instructional model, which left out many interactions we observed that were not directly related to formulating the answer for a specific task. This framework was missing the nuance to characterize the social interactions among students since it only focused on argumentation building.

Overall, these studies demonstrated how multi-level analysis of student talk can inform our understanding of the way in which students’ interactions influences their way of thinking through science problems (Coulthard, 1992; Kaartinen & Kumpulainen, 2002; Sampson et al., 2011; Sampson & Clark, 2011; Sinclair & Coulthard, 1975). However, rather than focusing SIDM solely on the viewpoint of instructor interaction, argumentation construction, or students’ explanation-building process, we aimed to develop a framework to characterize the nature of student engagement across an entire conversation that could be used to investigate a variety of student interactions.

Review of discourse analysis visualization methods

Reports have been published regarding the visualization of students’ discourse patterns, but most use tabulated frequencies of discourse moves to gain these insights, which is not a descriptive representation of the flow of discourse. Recently, within science education, social networking analyses have been used to characterize and visualize changes in student participation and interaction over time (González-Howard, 2019; Ryu & Lombardi, 2015). Although this method can illustrate who partook in the discussion, how often they participated, and who they responded to most often, these maps rely on the frequencies of interactions and lack the nature of the utterances within the discourse (González-Howard, 2019).

More informative methods of illustrating specific student talk patterns include mapping out speaking sequences as either strings of letters or the use of a graphical coding system (Keefer et al., 2000; Ryu & Sandoval, 2015). Specifically, to explore differences in decision-making and scientific argumentation patterns, Ryu and Sandoval (2015) chronologically mapped students’ speaking sequences using the following form: “an argumentation code (speaker)”. For instance, Dan’s explicit agreement to Jack’s claim was written as Claim (Jack) Explicitly Agreed (Dan)—C(J)EA(D). In a similar fashion of sequencing argument structures, Keefer et al. (2000), used a graphical coding system to map the logic of students’ arguments in peer-led student dialogues to identify what characteristics promoted a productive critical discussion. Different shapes were used to identify an argument, a concession, a challenge, or when an answer to a challenge was made, and arrows were used to identify how these elements were linked to each other (Keefer et al., 2000).

Although both studies illustrate differences in students’ discourse patterns in the context of argumentation, they do not capture the way students engaged in conversation prior to an argument nor what sustained an argument. Given that only a portion of student’s scientific discourse is focused on creating arguments, capturing how students talk and respond to each other in discussion beyond the context of argumentation would be informative of what constitutes productive discourse within STEM classrooms.

Context

Data presented in this paper was collected as part of a larger study investigating student engagement in introductory chemistry courses that was conducted at a large Midwestern university in a large enrollment active learning classroom and associated discussion classrooms. Data for the examples used below was collected in an introductory chemistry course designed for students who did not have an advanced chemistry course in high school. The structure of the course included both lecture and discussion components. The lecture component met for 75-min periods twice a week and was led by the course instructor in a large auditorium setting. In lecture, students worked on activities in self-selected groups of 3–4 people for 20–30 min over the course of the class period. These activities were either from a Process Oriented Guided Inquiry Learning (POGIL) workbook (Garoutte & Mahoney, 2015) or questions delivered using a student response system. The discussion portion of the course consisted of multiple sections of 25–30 students, each meeting for a 50-min period once a week, led by graduate teaching assistants.

Methods

The data presented in this paper describing the development of the SIDM framework was obtained during weeks 4 and 5 when the topic of “Naming Ionic Compounds and Type I and II Binary Compounds” was discussed in the lecture and discussion settings. This study was IRB approved and participants gave informed consent.

Data collection and transcription

Groups for analysis were selected at random from those that had consented to participate in the study. Video recordings were collected for every lecture session throughout the semester. For this study, pseudonyms were used to maintain the confidentiality of the participants. Pseudonym genders are random and have no relation to the participants.

Portions of the videos where students were working on small group activities were transcribed verbatim using a naturalized transcription approach (Bucholtz, 2000). Additional physical cues, such as shaking head or eyerolls, and tonal cues were noted in brackets to help convey the intent of an utterance if it would otherwise be unclear. For example, if a participant responded with “shut up”, it was noted if the intent of this utterances was truly meant as a directive to stop talking or as an expression of surprise or shock. This was based on voice inflection, context of conversation, and the colloquial slang of the community in which this study took place.

Developing the student interaction discourse moves framework

Coding student group interactions

Existing frameworks developed by Sinclair and Coulthard (1975), Coulthard (1992), Kaartinen and Kumpulainen (2002), Sampson, Grooms, and Walker (2011), & Sampson and Clark (2011) served as a starting point for the development of the SIDM but were too narrow in scope, focusing only on one-on-one interactions or specific aspects of discourse such as argumentation and explanation building. Developing the framework started by recording observational notes on students’ actions and their interactions based on video recordings to account for all the interactions of participants. While our focus was on student moves while they were working through an assigned task, other interactions was also noted, which led to the development of the Type of Interaction codes first.

The next level describes the Primary Intent or purpose of student speech when they were working through an assigned task. While many of the discourse moves presented in previous studies were exhibited by our participants, additional discourse moves were observed in the data. It was also the case that even when the same labels are used, the definitions are different since they are not restricted to the narrow contexts of those studies. For example, the discourse moves initiating, commenting, and concluding share the same names as those from the (2002) framework, but the definitions are different since they are not restricted to reasoning and can be the Primary Intent for any Nature of Utterance. Moves of contributing to discussion and questioning were characterized in the (2011) framework, however, whereas in their framework these moves were one code, these ideas were separated into their own codes for the SIDM framework. These moves were separated because it was observed that questions prompted more explanations by students, impacting the flow of student discourse differently than students simply making a statement.

The third level of the SIDM framework characterizes the manner in which students engage in a specific discourse move and are classified as the Nature of Utterance. For instance, a student could pose a question to clarify an idea, request information, or seek reasoning for an answer. As such, moves listed under the Nature of Utterance are not exclusive to one Primary Intent. This allowed moves such as revoicing an activity prompt or information processing the ideas presented in the prompt to be associated with different discursive moves such as initiating or contributing to discussion. Some of the moves characterized in the Nature of Utterance codes such as assessing, building, clarification seeking, explanation seeking, information seeking, reasoning, rebutting, and reporting have also been reported in previous studies (Kaartinen & Kumpulainen, 2002; Sampson & Clark, 2011; Sampson et al., 2011), but these studies did not use them as codes in the same way we have done here.

To account for the full range of interactions we observed, open coding to form axial codes as described in Merriam and Tisdell (2015) was used to develop the codes at each level (Merriam & Tisdell, 2015). The original scheme was used to code segments of transcripts from both the setting addressed in this paper along with transcripts from other settings within a larger project that emerged from this initial study. From here, new axial codes were proposed to the current list of codes to better characterize specific moves. This iterative process continued until no new codes were needed to characterize the student interactions. Part of this iterative process included creating tiers. Using tiers as a structure for SIDM was inspired by previous frameworks, to better organize the codes of our framework. Creating a tiered analytical framework was done to minimize the number of discourse move codes as many moves can be used in a variety of speech types such as questioning or commenting. The tiered nature of the framework also provides the ability to characterize a conversation at different levels depending on the nature of analysis.

Results

The resulting SIDM analytical framework can be seen in Fig. 1 and is composed of three levels: the the Type of Interaction, the Primary Intent, and the Nature of Utterance; each of which is denoted in underlined italics throughout the paper. In the next sections we will describe each of the code categories along with key features. More detailed information on all the codes along with their key features can be found in the Supplemental Information. The SIDM framework allows for analysis of discourse from the students’ perspective and the applied codes characterize student interactions while they are working on classroom activities.

Fig. 1

Student Interaction Discourse Moves three-tiered analytical framework

Student interaction discourse moves analytical framework

The first layer of the framework, Type of Interaction, characterizes how students broadly interact with each other over a given period and has subcodes denoted in lowercase italics throughout the paper. Definitions for Type of Interaction codes can be found in Table 1. At this level, we identified when students were working independently, interacting with the instructor, conversing about the task, conversing off-task about the course in general or off-task about their personal lives, and finally if the students were not engaged at all with the class. Accounting for these types of interactions as a first tier provides a broad sense of the social dynamics within a group and helped identify instances to code with the more detailed tiers when students were engaged in completing tasks.

Table 1 Definitions of student interaction discourse moves type of interaction codes. Abbreviations of each discourse move are in parenthesis next to the name of the code

The second level of our framework, Primary Intent, described the purpose of student talk and captured the temporal nature of a conversation during on-task interaction periods. Definitions for the Primary Intent codes can be found in Table 2. The list of Primary Intent codes are denoted using lowercase italics throughout the paper.

Table 2 Definitions of student interaction discourse moves primary intent codes. Abbreviations of each discourse move are in parenthesis next to the name of the code

The temporal component of the Primary Intent is best illustrated in the codes of initiating and concluding, as these types of utterances are usually the end caps to a conversation. Questioning occurs anytime someone asks a question, whereas the contributing to discussion and commenting occur whenever an utterance related to completing the task or judgement of the task is made. These codes are directly related to the exchange of information and ideas. Managing occurs whenever an utterance about roles in the group are made; this code is related to productivity in small groups as it relates to how effectively students are able to work together to complete a task (Cohen, 1994; Lemke, 1990). A code related to the social dynamics in the group is acknowledging, which characterizes the utterances that were made in response to another person’s utterance but did not add new content to move the conversation forward. External interaction is used to characterize when a student from another group interacts with the group of focus. Accounting for the intent behind student speech as a separate tier from the Type of Interaction allows researchers to characterize broadly what function the student is providing to the group’s conversation, without getting into the nuanced characterization seen in the Nature of Utterance.

The third level, Nature of Utterance, characterized the ways in which students display a specific Primary Intent and are denoted in lower case italics throughout the paper. Definitions for Nature of Utterance codes can be found in 3. These moves were used by students in a variety of contexts and thus can be applied under any Primary Intent.

Table 3 Definitions of student interaction discourse moves nature of utterance codes. Abbreviations of each discourse move are in parenthesis next to the name of the code

Some codes were primarily associated with a single Primary Intent, such as clarification seeking, explanation seeking, and information seeking with questioning moves or presenting a claim, procedural, providing information, reasoning, rebutting, and rejecting with contributing to discussion moves. Others were associated with a wider range of moves, such as activity prompt, agreeing, assessing, building, information processing, motivating, organizing, past experiences, personal remark, repeating, reporting, summarizing and nonverbal interaction. Accounting for these moves as a separate coding tier from the Primary Intent allows researchers to capture a nuanced view of student interactions without creating an unwieldy number of codes.

Applying the analytical framework to transcripts

To illustrate the application of this framework, excerpts of transcribed and coded discourse from the lecture group and discussion Team 1 have been provided and can be found in Tables 4 and 5 respectively. The three tiers of the framework were applied sequentially to a transcribed group conversation: coding started with the broadest coding category, Type of Interaction, and ended with the most detailed, Nature of Utterance. Each category was applied to an entire transcript before the next category was coded. When applying codes from this framework, two units of analysis were used; ‘conversational turns’ and ‘utterances.’ The first unit, ‘conversational turn’ (referred to as ‘turn’), and always denoted in single quotations throughout the paper, was defined as every time a different person began speaking.

An example of a conversational turn can be seen in Table 4, Line 1 where Blossom begins the group conversation with a ‘turn’ of reading the activity prompt followed by Mario starting a new ‘turn’ on Line 2 with “I don’t know”. In some cases, ‘turns’ would have more than one Primary Intent. For this reason, the second unit of analysis, ‘utterance’, always denoted in single quotations throughout the paper, was created, breaking up a ‘conversational turn’ to capture changes in Primary Intent. This breakdown was done in one of two ways, the first being by sentence. An example of this can be seen in Table 4, Line 4 where Mario’s first sentence was reading the activity prompt, but their second sentence was asking a question. However, not all ‘utterances’ are straightforward sentences, but rather can be fragments with separate meanings. An example of this can be seen in Table 4, Line 12 where Blossom first is presenting a claim, but then second-guessed themselves in an assessing nature, creating two separate ‘utterances within their conversational turn.’ In this instance it does not make sense to have a researcher choose between presenting a claim and assessing codes for the sentence because how those two moves effect conversation varies. Mario may not have corrected Blossom in line 13 if Blossom had not expressed uncertainty. Creating a smaller unit of analysis in utterances allows a researcher to capture all the nuance in student speech so confounding variables in conversation coding is minimized. No matter the unit of analysis, each unit of analysis had a singular Type of Interaction, Primary Intent, and usually a single Nature of Utterance. However, there are three instances where multiple Nature of Utterance codes could be applied for a single utterance. The first occurred when using the building code. This code is always co-coded with another Nature of Utterance code to characterize the nature of the expansion or completion of the ‘utterance’ or ‘turn’. The second is the non-verbal interaction code, which is applied similarly to that of the building code. This code is dual coded with another Nature of Utterance to characterize if a ‘utterance’ or ‘turn’ is that of a physical nature more than that of a verbal one. The third, however, occurred when a ‘turn’ or ‘utterance’ was too difficult to further separate into units to characterize with a singular Nature of Utterance code. An example of this can be seen in Table 5, Line 3, where Mario was interchangeably information processing and presenting a claim using single words to complete the given task. This kind of statement was rarely seen amongst the data sets and coding in this manner should be used sparingly as it can complicate interpretation.

Visualization of student discourse

The student discourse maps allow for the analysis of patterns in student interactions. The discourse map serves as a chronological visual representation of the verbal moves each speaker engaged in as the team worked through an activity prompt. The characterization of these student conversations was achieved by organizing the flow of the discourse map by the small group activity prompts in the order in which they occurred. Visual elements used in the construction of these maps are shown in Fig. 2. Discourse maps were only used with on-task discourse, unless notated with a pennant shape and labeled with the specific Type of Interaction. Speakers were represented using different colors and the Primary Intents of their utterances were represented using different shapes. The Nature of Utterance codes that align with each Primary Intent was also noted in these maps as a notation within the shape. Numbers were added to the top left corner of each shape within the figures to indicate which ‘turn’ corresponds to which shape or set of shapes.

Fig. 2

Key of colors, symbols, and markers used to construct the discourse maps

Example discourse maps constructed using the conversations from Tables 4 and 5 can be seen in Figs. 3a and b, respectively. If the discourse of a group is solely based on ‘turns’, the group’s discourse map will look like the beginning of Fig. 3a numbers 1–3. Each speaker has a color, a shape, and a Nature of Utterance per their ‘turn’. However, as stated before, conversations are not always linear, and certain ‘turns’ have been further broken down into ‘utterances’. When a single ‘turn’ was composed of multiple ‘utterances’, each ‘utterance’ received a shape that coincided with the coded Primary Intent and Nature of Utterance. These shapes were then stacked on top of each other, with a slight indentation to the right going down the stack of shapes to indicate the order in which the ‘utterances’ occurred. An example of this can be seen in Fig. 3a number 4, where Mario first contributed to the discussion by reading the activity prompt indicated by the green rectangle, followed by asking an explanation seeking question indicated with a green oval. When multiple Nature of Utterances were coded for a single ‘turn’ or ‘utterance’, one shape would be drawn with the Nature of Utterance codes stacked vertically within the singular shape. An example of this can be seen in Fig. 3b number 2, where Mario contributes to discussion by both presenting a claim and information processing interchangeably. These shapes were then stacked on top of each other, with a slight indentation to the right going down the stack of shapes to indicate the order in which the ‘utterances’ occurred. An example of this can be seen in Fig. 3a number 4, where Mario first contributed to the discussion by reading the activity prompt indicated by the green rectangle, followed by asking an explanation seeking question indicated with a green oval. When multiple Nature of Utterances were coded for a single ‘turn’ or ‘utterance’, one shape would be drawn with the Nature of Utterance codes stacked vertically within the singular shape. An example of this can be seen in Fig. 3b number 2, where Mario contributes to discussion by both presenting a claim and information processing interchangeably.

Fig. 3

Discourse map of Tables 4 and 5 excerpts. Panel a is from Table 4 and Panel b is from Table 5. Numbers at the top left corner of each shape correspond to specific lines from the excerpt. Color identifications of students:  , ,

Other annotations were used when creating these discourse maps. First, when multiple students were speaking at the same time; their correct colors, shapes, and descriptions were stacked to convey that no specific order could be determined based on the transcript. Second, arrows link statements that did not directly follow each other, such as when multiple conversations were going on between group members or when an agreement was made to an utterance much earlier in the conversation. Third, other markers such as //, ^, and * were used to represent nuances in students’ contributions as indicated in Fig. 2.

Table 4 Example Coding Excerpt. Coded excerpt transcript of students Mario, Blossom, and Harry as they worked through a POGIL prompt. Codes are written in the format as [Type of Interaction - Primary Intent – Nature of Utterance]

Reliability

After code saturation was achieved, inter-rater reliability (IRR) was calculated for 10% of the data set between the first two authors. Using a similar method of unitization as reported in literature, author one pre-segmented the randomly selected transcripts prior to both authors coding to reduce coding complications when segmenting ‘utterances’ from ‘turns’ (Campbell et al., 2013). The coding scheme was then applied to the segmented transcripts starting with Type of Interaction and working towards the most detailed layer, Nature of Utterance. Using qualitative data and value analysis software, MAXQDA 2020, the authors found an overall kappa value of 0.82, which according to McHugh (2012), falls into the strong level of agreement (VERBI Software, 2020). The kappa values for each layer of the coding scheme were also calculated and reported as follows: Type of Interaction (к = 0.97), Primary Intent (к = 0.79), Nature of Utterance (к = 0.65). Based on these values the broadest category produced an almost perfect level of agreement and the more detailed layers both fell into the moderate level of agreement (McHugh, 2012). For transcripts presented in this paper, any remaining coding disagreements within the presented transcripts were resolved, resulting in 100% negotiated agreement.

Table 5 Example Coded Excerpt. Coded excerpt transcript of Mario’s contributions as they worked through a worksheet prompt with Luigi (who does not contribute during this segment). Codes are written in the format as [Type of Interaction-Primary Intent – Nature of Utterance]

Discussion

The primary motivation for the development of the SIDM framework was to provide an approach to characterize the ways in which students interact while working together to complete classroom tasks. The goal was to be able to capture the flow and patterns of student interactions through the lens of social discourse. These interactions extend beyond just the reasoning or argumentation students engage in while working through prompts, including visualizing who is contributing to the conversation and in what ways. While graphs and tables provide information regarding the nature of social interactions and help identify differences in discourse moves among student groups, they are only summative in nature. They lack the ability to depict which utterances stemmed from which student and provide limited information on how students respond to each other to achieve a collective understanding. To address these issues and expand the avenues for analysis, discourse maps can be used. These maps chronologically illustrate the flow of students’ conversations and help analyze student crosstalk without the need to reference multiple graphs. As there are many features that compose these discourse maps, interpretation depends highly on the theoretical framework selected and purpose of one’s investigation. In the following section we illustrate how student discourse characterized using SIDM can be interpreted using different lenses to gain insights into understanding student engagement in collaborative learning environments.

Characterizing student engagement more accurately than frequency counts allow

As stated in the introduction, relying solely on frequency for the characterization of scientific discourse can be problematic. In this section we will provide a few instances of how tabulating frequency alone can easily misconstrue the nature of student conversations, as well as illustrate a more in-depth way to characterize student discourse that captures the nuance of student interactions. In this example, the students in this group were allocated 19 min to complete the POGIL activity (Garoutte & Mahoney, 2015) during lecture and respond to questions via the student response system. Analysis focuses on the Primary Intents and Nature of Utterances because only on-task interactions or unengaged interactions were seen in these settings. The Primary Intent frequency counts for students Mario, Blossom, and Harry are shown in Table 6.

Table 6 Frequency counts of Primary Intents for Mario, Blossom, and Harry in the lecture classroom

Mario and Blossom contributed at least double the number of utterances' when compared to Harry, who only had 25 coded utterances over the course of the class period. Frequency analysis could indicate that Harry’s contributions to the conversation appeared to be unsubstantial. However, upon a closer look, the nature of Harry’s contributions was important in advancing the group’s conversation forward. As seen in Figs. 4a and b, Harry often participated by challenging and questioning through rejecting statements and clarifying questions, which can help student groups deepen their understanding of concepts. Without exploring the nature of Primary Intent, the initial analysis would have indicated that Harry did not contribute much to the conversation. However, his contributions helped push the ideas forward, so through a social constructivist lens Harry was aiding in the interactive nature of the group discourse.

Fig. 4

Nature of contributing and questioning moves. Nature of contributing to discussion (a) and questioning moves (b) stated by students Mario, Blossom, and Harry in the lecture setting

Using framing theory to investigate the effects of facilitation strategies

Originating in anthropology, framing theory describes the way in which material is presented to a group influences the choices people make to process and engage with that information (Foxman & Bateson, 1973). People frame contexts, based upon their previous experiences, to determine what actions they should engage with in those settings. Students may frame “clicker questions” delivered during lecture as an opportunity for group sense making or as a time to use a formula to solve a problem (Tannen, 2009). This can be analyzed by looking at students’ verbal interactions. Students can move through more than one frame during an interaction, which can be seen readily through the symbols and colors used when presenting discourse maps. Discourse maps were used to see the flow of student interaction frames used during group discussion in lecture.

The discourse maps in Fig. 3 illustrate examples of two frames that students have used in the same day. Figure 3a illustrates the students working on a problem where they must submit an answer using a student response system where answers are compiled and discussed by the instructor afterwards. During this question, all students interacted by contributing to discussion which can be seen readily in Fig. 3a where all three colors are present. Harry rejects statements at lines 7 and 11, demonstrating that the group is not willing to settle on the first answer given just to submit something. Additionally, Blossom and Mario both ask questions that require further explanation of claims presented. This shows that the group’s frame during this question was to not only submit an answer, but to also understand the concepts asked in the question.

In the example shown in Fig. 3b, the students are working on a question in their POGIL workbooks. In this case, each student records an answer in their own workbook and there is no external check of their responses in contrast to the example above. Here, we see an interaction that where Mario shares his response, but there is no further interaction from the rest of the group. This shows that the students’ frame for this question was to just record an answer rather than engage in a more comprehensive discussion of the material.

The differences in frame for this brief two question analysis highlight the utility of using framing theory to investigate how students can move through different frames throughout a single class period for different questions. Similar analysis could be done in classrooms to reveal what types of facilitation methods or questions promote interactive framing. The claims presented for these two questions are not summative of the semester interactions and as such cannot be used to make broad claims about student framing during worksheet and clicker questions. However, this analysis illustrates an avenue by which student discourse moves can be analyzed through the lens of framing.

Using community of learners to investigate discourse moves

Community of Learners (CoL), a theoretical framework and teaching philosophy developed in K-12, strives to promote learning built on student-directed inquiry and participant collaboration. Built on five main pillars, CoL as a theoretical framework provides a guide for the interactions to focus on while investigating discourse in active-learning settings. Insights from CoL could direct discourse research towards those interactions that are dynamic in nature, reflective, and supportive (A. L. Brown, 1994).

To provide a few examples of how this could be done, one pillar is Community of Discourse, which promotes that differing ideas should be encouraged to migrate through conversations by all participating members (A. L. Brown, 1994). Using the discourse map in Fig. 3a as an example, we can see everyone present in the group is sharing their ideas throughout the conversation based on the multitude of colors and shapes across the entire discourse map. This would be interpreted as a productive pattern of discourse in terms of the presence of a Community of Discourse given that no one is being left out or ignored, and everyone is contributing to moving the conversation forward.

Another pillar in CoL is that classrooms are settings for multiple zones of proximal development, meaning that though progress in understanding should be towards the upper bounds of the community, no one person will or should be the all-knowing individual. Everyone can learn from each other, and the more productive group conversations are those where ideas are coming from multiple individuals and are being constructed together (A. L. Brown, 1994). Looking at Fig. 3a, both Blossom and Mario are presenting claims, reasoning through their answers, and questioning each other’s ideas, supporting the idea that no one person is functioning as the all-knowing, and they are building their answer together.

Even though only two pillars are addressed with examples in this paper, all five pillars of CoL can be used to guide one’s discourse analysis, either collectively or through a singular pillar. Based on the two pillars discussed, this lens conveys that the conversation in Fig. 3a is productive, especially when contrasted to the conversation in Fig. 3b where Mario is the only one verbally responding to the task.

Implications for research

The Student Interaction Discourse Moves framework provides many opportunities for discourse analysis research to expand using both the tiered coding scheme and discourse maps. SIDM allows researchers to more readily incorporate both the cognitive and social sides of student interactions while they engage with group work activities. Some areas of research that could be expanded using SIDM are how social aspects of student discourse affects their use of reasoning and explanation building, or the role instructor interactions play in student social and cognitive interactions. Analysis of student discourse using SIDM could also be paired with other analysis looking at the effects of different types of questions on student discourse or how their social interactions affect their knowledge building across different content areas.

As stated previously, interpretation of SIDM coding depends highly on the theoretical framework selected and the purpose of the investigation given the detailed nature of the framework. Using the example analysis lenses provided in the discussion, we now give more specific implications in line with those theoretical frameworks. Additional analysis through the lens of framing theory could investigate what discourse moves initiate transitions of frames through analysis of primary intent shifts. Framing theory could also be used to analyze the role of instructional strategies on student frames across the three levels of interaction. In addition to using Community of Learners to investigate student discourse, other potential routes of investigation could include the instructor’s role in group conversations, the equitability of contributions from all group members, or reflective learning practices engaged by the group Overall, SIDM offers versatility in the way student discourse can be analyzed using a multitude of lenses.

Limitations

The researchers note that during their development of the scheme they found a coder’s level of understanding regarding the English language, specifically English slang, could lead to challenges in the interpretation of student discourse. To decrease any confusion and assist in the dependability of this scheme, descriptions were added to the codebook by use of key features (S1). The researchers recognize unconscious bias held by them may be reflected in their codes and interpretations of students’ conversations. Finally, interpretations of a student’s discourse are only as good as the transcripts provided. Due to poor audio data in some cases, some pieces of conversations were inaudible. As a result, interpretations made by the researchers were solely based on what was able to be transcribed and could influence the results presented in this report. However, the novel approach to characterizing and visualizing student discourse moves that reflects the nuance of student discussion should be generalizable to other settings as outside input and rigorous development was completed to minimize bias.

Conclusion

Several studies have focused on characterizing student discourse patterns to gain insights into promoting productive discussions in college chemistry classrooms (P. Brown et al., 2010; Moon et al., 2017; Repice et al., 2016; Young & Talanquer, 2013). We extend that work here with a three-level analytical framework that characterizes student interactions both broadly and in-depth and provides a visualization of students’ conversations during small group activities. The analytical framework was designed to characterize student interactions at varying levels – Type of Student Interaction, Primary Intent, and Nature of Utterance. This more detailed level of analysis allowed for the identification of characteristics of productive discourse using a variety of lenses and could be visualized using discourse maps. These maps were developed to visualize student discourse patterns beyond a summative approach of frequency counts, which can misrepresent the nature of students’ conversations.

Although a single discourse map is limited in providing insights into the content and quality of students’ contributions, the pathways for analysis shown in this paper demonstrated how mapping student discourse can serve as an additional qualitative method to determine how and when students engage in certain moves. This is important for researchers who are studying collaborative learning to achieve rich descriptions of student interactions and identify factors that promote interactions where students collectively process information, clarify and exchange ideas with each other, ultimately resulting in deeper understanding. Potential research foci that can be addressed with such a framework include exploring students group interactions while working through a task, documenting the ways in which students manage times of confusion or disagreements, and what factors influence students to seek help from an instructor.