Participating in the scientific publication process: exploring how pre-college students perceive publication within the scientific enterprise

Abstract

Scientists spend a substantial amount of their time engaging with the primary literature: reading, constructing, reviewing and revising it. Yet, the role of primary literature is generally absent from the development of scientific inquiry skills in the pre-college science classroom, thus undermining a true understanding of what it means to do science. In this study, we examined middle and high school student perceptions of scientific inquiry and the role of disciplinary literacy practices after engaging in scientific review and publication of their research papers. We interviewed twelve students who published their papers in the Journal of Emerging Investigators, a science journal dedicated to publishing the research of middle and high school students. Students acknowledged the important roles that effective communication, scientific review, and revision played in their research projects. Further, after engaging with professional scientists through the scientific review process, students expressed increased confidence and belonging as a scientist. However, students primarily viewed the writing and publication processes as personal endpoints for their projects rather than an integral part of all stages of scientific inquiry and knowledge construction. If students are to develop an understanding of how reading, writing, and peer-review are critical pieces of doing science, then our work suggests that disciplinary literacy practices should be explicitly discussed and included in all parts of the research process. While not all students will be motivated to publish their research, our work has important implications for integrating disciplinary literacy practices into student scientific inquiry.

Introduction

It has become increasingly clear that the way scientists approach “doing science” is not fully reflected in classroom practices or by the curriculum standards and assessments for K-12 science education (Chinn & Malhotra, 2002; Phillips & Norris, 2009; Wong & Hodson, 2009). Specifically, the role of communication (reading and writing) within scientific inquiry is emphasized much differently in the scientific field compared to the classroom. For scientists, communicating one’s work is not just a critical output of their research, but an integral part of the entire process from initial observation to hypothesis development, design and revision of methodologies, results interpretation, and finally integration of conclusions into a larger knowledgebase (Halliday, Martin, & Martin, 2003; Tenopir & King, 2004; Yore, Hand, & Florence, 2004). As eloquently put by Norris and Phillips (2003):

Reading and writing are inextricably linked to the very nature and fabric of science, and, by extension, to learning science. Take them away and there goes science and proper science learning also, just as surely as removing observation, measurement, and experiment would destroy science and proper science learning (p. 226).

Beyond the acts of reading and writing, the process of peer-review engages scientists in a constructive and communal knowledge-building endeavor, which is meant to ensure the accuracy of the science and improve the scientific underpinnings of the paper. The multiple and integrated ways in which scientists use, and engage in, communication are collectively referred to as STEM disciplinary literacy skills, and it is these skills, we argue, that should be explicitly developed in young scientists. It becomes clear by observing the practices of scientists that disciplinary literacy should be part of programs and curricula that seek to engage students in authentic science inquiry.

The practice of communication within the scientific classroom, however, diverges significantly from that of professional scientists (Falk & Yarden, 2009). A starting point for this critique is the texts used within classrooms, which are almost never authentic primary sources (Chiappetta & Fillman, 2007; Phillips & Norris, 2009). Unlike primary literature, textbooks do not reveal the nuances of scientific argument and the many sources and pieces of evidence used to construct a single argument (Binns & Bell, 2015; Penney, Norris, Phillips, & Clark, 2003). Furthermore, the classroom practices of inquiry, which primarily focus on the experimentation, generally come from textbooks or associated lab books and are not built on an explicit framework of primary literature and communal knowledge-building that occurs through peer-review.

The notion of incorporating authentic practices of inquiry, rather than strictly scientific content, into science education dates back to the early twentieth century, when John Dewey advocated that K-12 science teachers teach students the scientific method (Barrow, 2006; Dewey, 1910). In 1957, the Soviet Union’s launch of the Sputnik satellite prompted the National Science Foundation to focus on science and mathematics curricula, with the goal of teaching basic principles through experimentation. In the 1960’s, Joseph Schwab highlighted that traditional courses tended to teach “only the outcomes...divorced from the data which support them and the conceptual frames which define—and limit—their validity” (Schwab, 1960). Schwab believed that science should be taught the way it was conducted: including discussions and interpretation of data, and the process of engaging in original work (Schwab, 1962; Schwab & Brandwein, 1966). Unfortunately, although there was a call to include inquiry in the science classroom during the 1950’s to mid-1960’s, this call had mixed results and declined by the 1970’s and 1980’s. Various reasons drove this decline, including that teachers felt unprepared and unsupported, found inquiry difficult to manage or dangerous in the classroom, and saw it as confusing for students (Costenson & Lawson, 1986; Eltinge & Roberts, 1993; Welch, Klopfer, Aikenhead, & Robinson, 1981). Additionally, state education agencies typically issued goals that emphasized content and not inquiry, adding pressure to revert back to traditional courses (Welch et al., 1981).

The 1990’s and early 2000’s saw a renewed push for inclusion of inquiry in K-12 science classrooms within the United States (Barrow, 2006; Crawford, Krajcik, & Marx, 1999). This led to the Next Generation Science Standards (NGSS), which have put renewed emphasis on inquiry and analysis practices within the United States’ science curriculum (NGSS Lead States, 2013). Within this push, a number of frameworks have been developed to facilitate inclusion of scientific communication (Dijk, 2014; Yarden, Norris, & Phillips, 2015). However, the emphasis on communication within scientific inquiry still does not fully reflect how scientists rely on multi-textual communications, including the communal processes of peer-review, in building scientific knowledge. The NGSS include forms of communication (written, oral, discussion, diagrammatic, etc) as part of the 9–12 curriculum; these suggested communication practices are primarily used in limited ways that do not completely align with the ways that scientists use communication practices (NGSS Lead States, 2013). For example, within the practice of “Obtaining, Evaluating, and Communicating Information” there is no explicit description of using scientific communication to construct, inform and help answer a scientific investigation (NGSS Lead States, 2013). This is in direct contrast to how scientists use scientific literature in all parts of investigation, from observation and hypothesis development, to method construction and conclusions (Emerson, 2016). The introductory statement from NGSS regarding the practice of “Obtaining, Evaluating, and Communicating Information”, as well as the language of the practice, implies that the practice itself is meant to help students develop scientific literacy, not necessarily use the skills of reading, writing and review as part of their knowledge construction (NGSS Lead States, 2013):

Being able to read, interpret, and produce scientific and technical text are fundamental practices of science and engineering, as is the ability to communicate clearly and persuasively. Being a critical consumer of information about science and engineering requires the ability to read or view reports of scientific or technological advances or applications (whether found in the press, the Internet, or in a town meeting) … (page15).

Even recent assessment tools for scientific inquiry further demonstrate that the skills of communication are not considered an integrated part of scientific inquiry. While these tools assess analytic skills and tenets of inquiry theory, they do not explicitly assess students’ understanding of the complex and integrated roles of communication in building scientific knowledge (Deane, Nomme, Jeffery, Pollock, & Birol, 2014; Lederman et al., 2019). The widely cited assessment tool “Views about Scientific Inquiry” is able to look at nuanced understandings of science inquiry (Lederman et al., 2014). However, this assessment does not include explicit reference to the various ways in which communication is critical to science inquiry. Part of this omission may be due to the fact that scientists largely use, and communicate through, the primary literature in their work, which may not be as amenable or accessible to the K-12 STEM classroom (Tenopir & King, 2004). Thus, the lack of primary resources accessible to students, lack of teacher awareness and comfort with primary literature, or lack of time to devote to authentic literacy sources could contribute to the continued overall neglect of authentic communication practices in the STEM classroom (Moje, 2008; O’Brien, Stewart, & Moje, 1995; Pearson, Moje, & Greenleaf, 2010). We posit that excluding the primary literature process within inquiry may undermine how students understand scientific knowledge construction.

Scientific journals written by and for K-12 students, and particularly secondary, audiences may provide more accessible bridges to the primary literature for both students and teachers, much as undergraduate journals have done so in college settings (Stone, Jensen, & Beech, 2016; Tatalovic, 2008; Walkington, 2012). One recent example of this approach is the Journal of Emerging Investigators (JEI), an open access journal that publishes the research papers written by middle and high school students (Fankhauser & Lijek, 2016). At the crux of JEI is a structured scientific review process in which graduate students provide constructive feedback to student scientists throughout their journey to publication; thus, the publication process provides students an opportunity to engage in an aspect of scientific inquiry that is otherwise inaccessible to them. Given that this authentic communication approach is modeled after the process many professional scientists go through, we were interested in how the student authors of JEI experienced scientific publication. Specifically, we wanted to understand how these students viewed scientific inquiry and the role of publication within scientific inquiry. Here we report on interviews with twelve student authors about their experiences doing research, engaging in scientific review and publication, and their conceptions of these processes as part of the scientific enterprise. Our work suggests that while these students view the scientific review process as collaborative and constructive to their research projects, they still hold limited understandings of the general role of primary literature in the construction of knowledge. Based on our findings, we propose that including authentic disciplinary practices of scientific review and publication in scientific inquiry projects may be helpful to students in developing a better understanding of their own research and science as a social enterprise; however, if the goal is to help students develop a broader understanding of the role of communication practices in scientific inquiry, then more explicit discussion and inclusion of such practices in all parts of the research process is needed.

Theoretical framework

The broad, and sometimes misunderstood, concept of “scientific inquiry” is at the foundation of our research. In its most fundamental sense, scientific inquiry is the process of systematic investigation that scientists employ to investigate the world around them (Lederman, Lederman, & Antink, 2013). As described earlier, disciplinary literacy practices are fundamental to science inquiry, yet they are a typically neglected piece of the science inquiry experience for pre-college students. Our framework places disciplinary literacy practices within authentic science inquiry as essential components. In our model, we highlight the role of community-based analysis through peer-review and revision, which requires contextual supports, appropriate expectation and assessments, and engagement in forms of communication that include peer-review, i.e. primary literature (Fig. 1).

Fig. 1

Model of scientific inquiry. Blue arc highlights the stages of scientific inquiry that use experimental practices. The peach arc emphasizes the stages of inquiry that require disciplinary literacy practices. The width of the arcs represents the level to which the practices are used

Practicing science inquiry in the pre-college classroom

What the term “science inquiry” encompasses in terms of understandings and practices is broad, and, simultaneously, narrowly experienced in the K-12 curriculum. Since Schwab’s work in the 1960s, contemporary views of scientific inquiry have been refined. Today, debate still exists on the exact definition of scientific inquiry, but Lederman et al. (2014) most precisely defined what students should know about scientific inquiry with eight specific tenets. These aspects of contemporary scientific inquiry go beyond broad experimentation and focus on developing students’ understanding of the various mechanisms that scientists use to construct knowledge. The tenets include, for example, the development of the experimental idea and plan, the concept that various methodologies could be employed to investigate a question, and the relationship between data and evidence and drawing appropriate conclusions (Lederman et al., 2014).

As described earlier, in the United States, the suggested high school curriculum, NGSS, encourages incorporating scientific investigations and specific inquiry practices in the classroom or laboratory, but these practices constitute only a small part of scientific inquiry and lack interconnection. Consequently, while students may be engaged in evaluating scientific information (Practice 8), this may be divorced from their own inquiry-based project (Practice 3). The typical school investigation omits the rich discourse used to develop, assess, validate and evaluate one’s research project. As shown in their 2019 study, over 50% of evaluated seventh-grade students held naïve views on science inquiry tenets (Lederman et al., 2019). Thus, although students may be consistently engaged in doing experiments (only one part of science inquiry), there appears to be a disconnect as students can do the experiments without understanding the processes used to construct knowledge and the tenets of scientific inquiry set forth by Lederman et al., 2014 (Lederman et al., 2019).

Given the challenges students have in demonstrating deeper understandings of science inquiry, investigating the contexts and ways in which students effectively learn the nature of scientific inquiry has increased in importance. There are various ways of incorporating guided-inquiry in the classroom or laboratory, from open-ended student-guided inquiry projects, to informal inquiry projects. Most relevant to the present study is what students learn by engaging in student-driven research projects, such as through science fair, independent, or mentor-guided research, as in apprenticeship programs.

Student-driven open-ended inquiry projects (in the classroom) and research projects (outside the classroom) both seek to engage students in authentic scientific inquiry in a way that is similar to that of scientists. Each of these types of projects exemplifies the theory of situated cognition (Brown, Collins, & Duguid, 1989; Roth & Jornet, 2013), which posits that knowledge is inseparable from the context where learning occurs. In order to know how to apply scientific inquiry, students should participate in the process itself. Students who engage in open-ended inquiry within the classroom develop research questions and procedures to analyze data, often working collaboratively with peers on their projects. Apprenticeship projects that are either extracurricular or classroom-based student-scientist partnerships (SSP) also aim to engage students in scientific inquiry via “learning experiences wherein a learner works with an expert mentor in authentic contexts” (Sadler, Burgin, McKinney, & Ponjuan, 2010 p. 235). By participating in the iterative process of developing questions and methods, students ought to better understand and apply the nature of scientific inquiry.

A review of empirical studies from apprenticeship programs revealed that overall, these experiences result in increased science career aspirations and student understanding about the nature of science (Sadler et al., 2010). However, research apprenticeships have varying impacts on student understanding of scientific inquiry. Longitudinal case studies of seven high school students participating in SSP programs revealed that the programs did not allow for student generation of research questions or data analysis, and ultimately did not change student ideas about scientific processes (Moss, Abrams, & Kull, 1998). In a separate study, Bell, Blair, Crawford, and Lederman (2003) showed that high school students who participated in an 8-week science apprenticeship program gained new skills in the ability to do scientific inquiry, but the program primarily reinforced conceptions students already had about their knowledge of scientific inquiry.

Regardless of the outcomes of authentic inquiry projects, broadly incorporating authentic inquiry into science classrooms is even more challenging, and instead laboratory texts attempt to inject parts of this process into laboratory studies (Vasconcelos, Torres, Dourado, & Leite, 2012). What has become a typical outcome is that students may do an experiment, but they lack an understanding of how the experiment was conceived, why the hypothesis is what it is, and how the results can add to the scientific knowledge base. Even when students are engaged in the most “authentic” inquiry projects they can still lack understanding of the basic mechanisms by which scientists construct knowledge. Thus, only engaging in select pieces of the inquiry process is not sufficient for developing comprehensive understanding of scientific inquiry (Schwartz, Lederman, & Crawford, 2004).

Scientific inquiry and disciplinary literacy

Disciplinary literacy mechanisms in science can be divided into three broad interconnected practices: 1. Reading literature to learn content, develop ideas, and construct new knowledge; 2. Writing literature to communicate new knowledge and stimulate knowledge construction; and 3. Community-based analysis of work to revise and strengthen scientific knowledge (Fig. 1). Simply put, the practices of disciplinary literacy, the ways in which scientists engage in their community through the reading, construction, and evaluation of scientific communication are essential to the construction of scientific knowledge and thus to scientific inquiry (Moje, 2008; Shanahan & Shanahan, 2012). Tenopir and King (2004) demonstrated the importance of these practices in scientists’ inquiry with the observation that, on average, scientists spend close to 60% of their time engaging in communication-related activities.

To do scientific inquiry, that is, to build scientific knowledge, a scientist is required to use the disseminated communication from other scientists, and thus the ideas, methods, and conclusions of other scientists. Therefore, it is not surprising that the average scientist reads 333 pieces of literature per year, with over 40% of these readings comprised of primary literature (Tenopir & King, 2004). Generally, scientists read professional literature (literature written by their peer scientists) for a multitude of reasons, including staying abreast of their fields, developing arguments, assessing new methods, changing research contexts or direction, and learning conventions to write their own literature (Bazerman, 1988; Yore, Hand, & Prain, 2002, Yore et al., 2004). Norris and Phillips contend that reading is a constructive knowledge-building process that involves inferring meaning, evaluating content, and integrating content with what the reader already knows in order to interpret the text (Norris & Phillips, 2003). However, contrary to the ideas set forth by Norris and Philips, it appears that, in general, scientists read for more pragmatic reasons and tend not to use language associated with the epistemic concept that reading is explicitly building knowledge (Tenopir & King, 2004; Yore et al., 2004).

As Keys posited, the disciplinary literacy practice of writing in a scientific genre is one that “ … promotes the production of new knowledge by creating a unique reflective environment … ” (Keys, 1999). Thus, the expert scientist understands that the purpose of writing goes beyond simple knowledge telling and includes knowledge transformation (Emerson, 2016). Through their writing, the expert science writer engages in evaluating data, making meaning, and constructing and revising argumentation (Bereiter & Scardamalia, 1987). Indeed, research has demonstrated that writing can serve to increase knowledge and understanding of the nature of science (Kelly, Chen, & Prothero, 2000). However, an analysis of seventeen scientists’ writing habits demonstrated that some scientists hold more tacit views of the role of writing in science (Yore et al., 2002). These scientists viewed writing primarily as knowledge telling, with the intention to inform and persuade a specific audience. In a larger survey analysis, the majority of scientists responded that they never or infrequently used writing as a tool for learning (Yore et al., 2002). A subsequent study in 2004 found that the scientists interviewed were able to express metacognitive awareness of learning through writing, specifically that writing helped improve clarity and understanding of their research (Yore et al., 2004). However, scientists in the 2004 study still maintained that the primary purpose of writing is knowledge telling. Thus, in opposition to the knowledge-building theories of reading and writing and the archetype scientist who reads and writes as a constructivist mechanism, it appears that not all scientists approach reading and writing with the same ideal epistemic lens. Emerson found that senior scientists were more likely than junior scientists to express learning through writing, but the numbers of scientists expressing this viewpoint were still low (Emerson, 2016).

What sets primary literature apart from other forms of scientific communication is the peer-review process. The peer-review process, which requires evaluating and revising one’s writing and scientific underpinnings, is a disciplinary literacy skill that not only assesses the validity of the communication and science of a paper, but also engages the scientist in a rich discourse with a community of peers. Peer-reviewed literature is generally considered the gold standard in the scientific world, with some referring to peer-review as the “backbone of modern science” (Bornmann, 2015) and the “lynchpin of academic life” (Eisenhart, 2002). Further testament to this, Yore et al. (2004) found that scientists generally view refereed publications as more reliable sources of scientific information. The goal of peer-review is ostensibly to improve argumentation, evaluate methods, provide alternative interpretations of data, and provide suggestions for general improvement; the outcome of peer-review is visualized by the multiple arrows stemming from peer-review that requires the scientist to return to, and potentially revise, earlier parts of the inquiry process (Fig. 1). Stichler (2011) described the most important qualification of a peer-reviewer as “… the simple willingness to share knowledge and experience (p. 45)”. Thus, knowledge construction and revision should be natural outcomes of this process. But more research has been done on peer-review’s achievement of accurate and unbiased science than on how scientists use the peer-review process as a tool for inquiry. Numerous recent papers discuss problems with peer-review, which include allowing questionable science to be published, bias and lack of inclusivity, lack of consensus among reviewers, and censorship of innovation (Atkinson, 1994; Bornmann, 2011, Eisenhart, 2002; Lăzăroiu, 2019; Schnell, 2018). Taken together, these suggest that peer-review may not always be viewed, or used, as the tool of inquiry that we expect. Very few studies have looked at the potential cognitive benefits of peer-review and whether scientists conceive peer-review as a constructivist tool of inquiry. In fact, we only found one paper that investigated the role of peer-review on scientists’ perceived-learning. In an analysis of scientists at a Canadian university, Yore et al. (2004) found that scientists view the review process as constructive to their paper, their research approach, and the interpretation of their data. However, these scientists did not express the idea that the review process and feedback from peer-reviewers were part of their knowledge construction as we would theorize. Collectively, the data suggest that there is a gap between the theory of disciplinary literacy within scientific knowledge construction and how scientists perceive the role of disciplinary literacy practices.

Disciplinary literacy in pre-college science

Although there has been more emphasis on authentic science inquiry, we observe that many inquiry pedagogies disconnect the practice of experimentation from disciplinary literacy. For scientists, aspects of scientific inquiry are developed, reinforced and continually practiced through their interactions with other scientists, and particularly through the cooperative nature of communicating their science through the primary literature. Despite the apparent gap between the theory and practice of disciplinary literacy within pre-college science inquiry, a few approaches have been designed to involve pre-college students in STEM disciplinary literacy practices. These include writing-to-learn instruction, reading-to-learn, peer-review, and various communication assignments like elevator pitches, oral presentations, and lab reports. Yet, these approaches are often used as a way to learn content or they are presented as a simple outcome or last step of doing inquiry (Bell, Bricker, Tzou, Lee, & Van Horne, 2012). Given the aforementioned ways in which the primary literature is essential within scientific inquiry processes, it makes sense to attempt to engage in, or introduce pre-college students to, the primary literature as part of the way in which scientists do inquiry. Schwab advocated for the use of original scientific papers as curriculum materials, explaining that “They afford the most authentic, unretouched specimens of enquiry which we can obtain” (Schwab, 1962, p. 73). Although Schwab recognized the challenge of introducing such sources in the classroom, he concluded that “full-scale use of original papers is feasible at the high school level” (Schwab, 1962 p. 78). In fact, work has been done on developing disciplinary literacy skills and bringing primary literature into the classroom. Specifically, adaptation of primary literature (APL) for the classroom has become a tool whereby teachers read and synthesize primary papers to support scientific discourse in the classroom (Koomen, Weaver, Blair, & Oberhauser, 2016; Phillips & Norris, 2009). It is unclear how widespread this practice is, but research has demonstrated that introducing students to this type of work provides an elevated level of engagement for students in scientific discourse (Yarden, Brill, & Falk, 2001). Further investigation of APL has shown that students have a greater understanding of scientific reasoning (Phillips & Norris, 2009).

APL serves as a promising bridge to involve students in reading information like a scientist (Norris & Phillips, 2003; Norris, Stelnicki, & de Vries, 2011). However, APL and other approaches leave out other key constructs of disciplinary literacy: engaging in the scientific community by using and creating authentic information like a scientist. Although important, approaches such as writing-to-learn, reading-to-learn, APL, elevator pitches, and lab reports do not afford students the opportunity to experience the reading, writing and revision processes within the construction of scientific knowledge. Most notably, these aforementioned approaches do not emphasize the critical communal discourse that occurs through the peer-review process, a key component of primary literature.

At the college level, the act of revising a paper has been shown to provide a scaffold for students to learn how to communicate in a way that is aligned with disciplinary norms (Walker & Sampson, 2013). Furthermore, the process of writing a paper for publication has been shown to enculturate graduate and post-doctoral fellows into a research community, but these benefits could be experienced at much earlier levels given the opportunity (Florence & Yore, 2004). To date, no research has looked at how pre-college students experience scientific review and publication and the potential impact of these experiences, which is unfortunate since publication could provide a rich opportunity for student learning as it has with undergraduate students (Stone et al., 2016; Tatalovic, 2008; Walkington, 2012). Moreover, the disciplinary literacy skills that are involved in peer-review and publication directly support the tenets of scientific inquiry (Supplemental Table 1). Yet, it is unknown if pre-college students’ views of scientific inquiry and the social enterprise of science could be impacted by participating in such authentic processes as scientific review and publication.

Rationale for the study

There is a gap between how disciplinary literacy should be part of learning and practicing science inquiry and how students are typically taught, and exposed to, disciplinary literacy practices. Currently, it’s not known how younger scientists perceive the role of disciplinary literacy practices, specifically the role of the primary literature, within science and how their views align with contemporary understandings of STEM disciplinary literacy and scientific inquiry. However, given the aforementioned gap, we would hypothesize that students would have under-developed or superficial perceptions of the role of disciplinary literacy skills within the enterprise of science. This small exploratory study is the first to investigate the pre-college student experience in constructing and publishing primary literature articles based on their science inquiry projects. We explore the experiences and perceptions of twelve pre-college students who have participated in a STEM inquiry and disciplinary literacy experience and have achieved something rare for pre-college students: a scientific publication. These twelve students submitted manuscripts through the JEI, received feedback on their science and communication from three graduate students in their scientific field through a structured-review process, revised their manuscripts based on this feedback, and eventually had their manuscripts published. We recognize that this experience may diverge from professional peer-review and publication; namely the review process has been intentionally designed to be supportive and is not performed by peers in the strict sense. The ethos of JEI is that both reviewer and student are scientists and thus peers. However, to avoid confusion we refer to the process as “scientific review” rather than “peer-review”. By exploring their experiences and perceptions, we hope to gain a better understanding of how these students view connections between scientific research and disciplinary literacy practices and whether their perceptions are aligned with contemporary views of scientific inquiry and disciplinary literacy. Our work seeks to inform future studies investigating pre-college practices and understanding of disciplinary literacy skills within the context of scientific inquiry. Thus, our work was guided by the following three questions:

• 1. How do students conceptualize their own research experience?

• 2. What are students’ perceptions of the disciplinary literacy practices of reading, writing, review, and publication within the enterprise of science?

• 3. What do students view as the value and outcome of scientific review and publication?

Methodology

Research setting

JEI is an open access journal, started in 2011, that publishes research by secondary school scientists (middle and high school). Since 2018, JEI has been publishing about 60 student articles annually; articles in any field of science, technology, engineering and math are accepted and authors come from across the globe. The JEI website provides extensive guides and models to help students through the writing of their manuscript (www.emerginginvestigators.org).

When student author submits a paper to the JEI it undergoes a pre-review phase to ensure that the basic components of a scientific article are present. Papers may be returned to the author with request for revision at this stage if there are major structural problems or missing sections. Once a paper makes it through the pre-review phase, it moves into the scientific review phase. The paper is sent to 3–4 reviewers, usually graduate students, in the field of the paper (Johnson & Fankhauser, 2018). Reviewers are requested to provide feedback on the following as applicable to a given paper:

• Required scientific changes

• Recommended scientific changes

• Required presentation (communication) changes

• Recommended presentation (communication) changes

• Required figure changes

• Recommended figure changes

Example reviewer comments, aligned to tenets of scientific inquiry, can be found in Supplemental Table 1. Following the scientific review, an editor summarizes the reviewer feedback and compiles a list of required changes before sending the editorial letter with the reviewer comments to the author(s) (an example editorial letter can be found in Supplemental File 1). Students are allowed as much time as needed to revise their paper and resubmit. The resubmitted paper is reviewed by the same editor, and in some cases sent out for scientific review again. Once the paper has no more revisions required, it is sent to a copyeditor and proofing editor to finalize the manuscript for online publication.

Participants

In March of 2018 an email was sent to student authors who had published a paper with JEI in the last 3 years. The email explained the nature of the study and requested a voluntary 60-min interview about the student experience in publishing a paper with JEI. Twelve students responded, representing approximately 10% of authors contacted. The emails were sent to the email addresses that students included in their initial submission. Many of the student emails were no longer active since the emails were associated with the previous academic institution of the student, which could factor into the low response rate. All twelve students were scheduled for an interview by Skype or Zoom (this included two siblings who co-authored a paper; thus, eleven interviews were conducted with twelve students). The authors represented a mix of genders (5 males, and 7 females), self-described cultural backgrounds, and grade levels, and their respective manuscripts were also across a diverse field of sciences (Table 1). Students over the age of 18 provided written consent; for minor participants the legal guardian provided written consent and the student provided verbal assent in accordance with the IRB approved by [institutions]. All names have been changed to protect the identity of the students, and where necessary, information has been redacted from quotes to avoid revealing identities.

Table 1 Interview participants, background and research environment

Study design

Taking a constructivist approach (Charmaz, 2006), the research team chose semi-structured interviews as the research instrument most able to elicit participants’ own meanings and perceptions for their publication experiences. Given how little is known about pre-college student scientific publication experiences, interviews afforded an important interactive approach that allowed exploration of authors’ experiences beyond the researchers’ preconceptions of what those experiences may have consisted of. The study also took a discourse-based interview approach, an approach designed to stimulate recall and elicit writers’ tacit knowledge (Odell, Goswami, & Herrington, 1983). The interview framework was developed by the first three authors—a scientist, writing researcher, and undergraduate research student—each of whom brought complementary expertise and research sensibilities. The scientist and first author, for example, brought a disciplinary sensitivity to science inquiry, whereas the writing researcher brought a lens focused on authors’ writing knowledge and processes. The open-ended questions developed addressed both perspectives and were categorized into four parts:

• 1. How did the student perceive the research and the publication process? This first part focused on the student’s general experience with their research and publication process. For example, we asked each student about any challenges they experienced.

• 2. How did the student experience manuscript development and revision? This part of the interview focused on the student’s specific paper and their experience in constructing and revising each part of the paper; we asked specific questions about choices in their manuscript, such as, “How did this reviewer’s comment impact your revisions to your results section? Did you consider putting X before Y?”

• 3. How did the student perceive their learning? For example, we specifically asked: “What do you think is the most important thing you learned by going through the publication process with JEI?”

• 4. What were the student’s academic and professional outcomes and goals? For example, “How do you think publishing with JEI has affected you academically?”

The interview questions, aligned with our research questions, can be found in Supplemental File 2.

Data collection

The interviews took place in summer of 2018. The first and second author trained two other authors (also scientists with expertise in science education) to lead interviews by providing the interview protocol, instructions on how to use discourse materials in the interview, a recording of the first interview to assist in the training, as well as key methodological literature (e.g., Charmaz, 2003). The interview protocol was semi-structured to allow for naturalistic conversation between the student and interviewer around preset topics, and afford deeper exploration of participants’ experiences (Charmaz, 2006). The young age of our interviewees also made this preferable as a naturalistic conversation built rapport and avoided slipping into “interrogation” (Charmaz, 2006). The semi-structured nature of the interview means that some questions were excluded in interviews if they were not relevant, while other questions may have been asked in a different order in response to participant statements, or the interviewer may in some instances have asked a clarifying question as a follow-up to a particular participant comment. For example, authors who had since graduated from high school were asked about how their publication experience may have factored into their transition to college, while this was omitted for younger authors. Prior to the interview, each student was emailed their published paper and the editorial letter and reviewer feedback to stimulate recall. The second part of each interview focused on specific choices students had made in their manuscript and was conducted as a discourse-based interview (Odell et al., 1983). This meant that a portion of the interview made specific reference to the participants’ discourse, which served as a common point of reference during the interview with questions about why authors made certain textual choices and not others. For instance, a reviewer comment may have encouraged a given student to revise their hypothesis, which the interviewer would have asked about during the discourse-based portion of the interview. The semi-structured nature of the interviews coupled with the students’ written discourse, allowed for consistence of topic among the interviews along with sufficient flexibility for in-depth exploration of authors’ individual publication experiences grounded in concrete choices and artifacts, such as through questions specific to authors’ editorial letters, reviewer feedback, and manuscripts (Odell et al., 1983).

Data analysis

Because little is known about student experiences in peer-review and publication, we took a grounded theory approach to analyze the data, an approach that sensitized us to emergent themes that were important to authors’ experiences but that we might not have anticipated beforehand (Charmaz, 2006). Each interview was transcribed and segmented by thought, at locations where the participant clearly changed idea, either because the interviewer moved on to a new question or because the participant obviously switched thoughts in their answer. Initial coding was performed collaboratively by three of the researchers in order to reach a shared agreement on what we were seeing in the transcripts and more fully benefit from our multidisciplinary perspectives (Smagorinsky, 2008). In a collaborative process, the researchers met and performed an initial reading aloud of segments and discussion about the ideas, practices, experiences, and meanings they reflected. Between meetings, the paper segments were divided among three of the researchers, with each researcher reading the segments and developing specific codes from their segments. Through several collaborative meetings, the three researchers discussed and refined these emerging codes, sorting segments and assigning segments specific codes using constant comparison—reading and comparing segments within and between categories to reach agreement on a reading of the data and emerging categories (Glaser, 1965; Smagorinsky, 2008). Disagreements were openly discussed among the three researchers and consensus from at least two of the researchers was required before assigning a category. Initial coding produced over 75 preliminary codes, which informed the first author’s emerging research questions related to scientific inquiry. At this point, the first author employed the qualitative data analysis software MAXQDA for focused coding, reducing the 75 codes into larger categories that related specifically to scientific inquiry and setting aside codes not connected with the line of inquiry reported here. Through focused coding, the first author identified and employed eleven codes relevant to scientific inquiry (Charmaz, 2006; Saldaña, 2009). Further refining and axial coding revealed the relationships between codes and larger thematic dimensions (or “axes”) of related codes (Charmaz, 2006). For example, the author identified two codes that were related to revision and recursiveness of science publication, which eventually became one code. This process resulted in the eight codes shown in Table 2.

Table 2 Coding definitions, instructions and examples

Once the finalized coding scheme was complete and applied to all the segments from all the interviews, another researcher coded the segments using the same coding scheme (Table 2). This resulted in a 95.4% inter-rater agreement across all the data and an average of 95% across each of the codes (McHugh, 2012). Cohen’s κ was run to determine the level of agreement between the two researchers, revealing almost perfect agreement between the two researchers’ coding decisions, κ = 0.946 (95% CI, .0.847 and 1; SE of 0.037).

Results

The findings below represent eight codes within our three research questions on student experiences of their research and JEI publication experiences (Table 3):

• 1. How did students conceptualize their own research experience? (2 codes)

• 2. What were student experiences and perceptions of the publication process? (3 codes)

• 3. What do students view as the value and outcome of scientific review and publication? (3 codes)

Table 3 Identified codes and example comments from student author interviews

How did students conceptualize their own research experience?

As part of our investigation, we were interested in how students approached and experienced their research, specifically in the design and experimentation phases prior to submitting a paper for review and publication. Across all interviews, students primarily focused on the complexities of starting their research projects and of deriving accurate and meaningful conclusions. These responses were captured in two codes:

• 1. “Scientific research requires operationalizing an idea”

• 2. “Scientific research results in nuanced conclusions which are not always conclusive”

The first code, “Scientific research requires operationalizing an idea,” reflected the variety of motivations and challenges for students as they started to investigate a specific problem in a focused and systematic way. Students discussed how they developed their initial ideas, with many indicating that they developed their ideas through an everyday observation or conversation with a mentor. For example, Michelle explained how a common experience led to her project idea: “I had heard of the very low efficiency of [object] and wanted to find out more and see how I could help it.” Similarly, Anvika used an everyday experience in developing her research idea: “He’s my younger brother, so, um, a lot of our life has been kind of built around his surgeries, and I’ve always kind of grown up wondering why he had this, um, condition.” And Vivek informed us: “And so I knew that when I embarked on my own investigation, my end goal was to hopefully, or eventually, find a specific biomarker of depression.” These students’ projects were guided by questions stemming from initial observations; these comments also appear to indicate that students saw science research as an avenue for addressing meaningful problems facing society.

To transform their ideas or questions into concrete plans, many students commented that their experimental design development came from resources such as other literature, conversations with teachers or parents, or emails with professors or other professionals. For example, Noah explained the difficulty in progressing from an idea to plan:

I’d say the hardest part is going from idea to operationalizing it. I always read these papers and I think of a crazy idea, but then I realize, oh, I need like a million-dollar grant to do it. That’s practically impossible. So, it’s a very large gap between an idea and operationalizing it. But you have to think a lot outside the box.

Noah’s comment starts to inform us on the student perspective of the “process of science,” suggesting that part of doing science for these students is figuring out how to do the experimental investigation. Ishita had a similar comment: “After I got that basic idea, I went from there and started researching about biofilms and reading papers and everything.” Noah’s and Ishita’s use of the literature, however, was exceptional among our interviewees and may have been conducted as part of a class or school-wide program that could have emphasized such processes. Instead, the majority of other students referred to using resources immediately at their disposal, such as a teacher, parent, or a relatively convenient tool or observation. For example, Vivek explained how he generated his initial idea based on computational tools his teacher had taught him:

She [teacher] taught a lot of computational biology rules and things like that. And then I would say, like, around the end of my first semester sophomore year, I decided to start my own investigation using those tools...

Similarly, Caroline explained the role of discourse with her teacher and parent in the process:

My question was a combination of a standard protocol in microbiology, transforming [organism] with [protein] … And so I was like, hm how can I combine these? … I don’t think I came up with that combination in particular, but I think either my teacher or my dad suggested something along the lines of that.

These comments show that while students eventually submitted their project for publication, and even remark on the value of published research in progressing science and improving research (see section below), few students actually used published literature during the initial step in their research process and instead relied on more common or familiar resources.

Beyond the initial step of operationalizing the project, students also focused on the difficulty of making accurate conclusions from the data and deriving meaning from these conclusions, captured in the code “Scientific research results in nuanced conclusions which may not be conclusive.” The comments in this code reflect revelations by students that conclusions are not necessarily concrete or conclusive. For example, Noah’s comment indicates a recognition that his result is tenuous:

So, when you’re going from proof to support, it’s instead of, um—it’s like correlation doesn’t mean causation. So, just because you get this correlation, doesn’t mean it will always hold [or] happen that way, but it shows that there might be some support between them.

Similarly, Rosa touched on the difficulty of making conclusions that are supported by the data:

You cannot make a statement that are not supported by your data. Like, I was saying, yeah, [subjects] are better at [specific ability]. No! My data suggest that because I only got significant differences in some.

Although Anvika acknowledged the tenuous nature of her conclusions, she recognized that even preliminary conclusions could make significant contributions to research conversations:

Because even our conclusion of our study was a very preliminary, um, conclusion. I mean, it was just defining of a pathway. It wasn’t gonna help us, you know, cure [syndrome] by any means. It was just kind of a discovery of a starting point of research.

These comments reveal that these students understood the complexity of analyzing data and developing meaningful and accurate conclusions. The students articulate larger ideas on the nature of data and evidence, and while students did not use exact terminology of “data” in regard to “evidence,” their comments indicate a recognition that their data is tentative and does not necessarily constitute unequivocal evidence or a concrete solution.

Together, these two codes reflect how students conceptualized the ends of their research projects. At the beginning stages of their projects, students exhibited less-informed approaches in developing their projects. Rather than seeking knowledge, instruction, or inspiration from the published literature, students relied on familiar resources, such as parents or teachers. They initially approached their research projects as novices, even though they eventually took much more advanced steps. As students discussed their conclusion-making, they appear to take on more informed perspectives that are aligned with views of scientific inquiry described by Lederman et al. (2014). Specifically, students like Rosa and Noah indicate a recognition that their data can be interpreted in different ways and that conclusions should be rigorously supported by data. A larger question remains, however, of whether these students can extrapolate their beliefs beyond their research projects to the operation of science as a whole.

Students’ perceptions of the disciplinary literacy practices of reading, writing, review and publication within the enterprise of science

We identified three relevant codes reflecting the challenges and revelations the students faced while going through the writing, scientific review and publication processes. Here we refer to the overall process, from writing to submission, revision, and online publication, as the “publication process” as students generally did not differentiate between these stages. The following three codes captured student experiences and perceptions of the publication process:

“Scientific publication requires revision and refinement”.

“Scientific publication requires effective and precise communication”.

“Scientific publication is a cooperative endeavor”.

The code “Scientific publication requires revision and refinement” was one of the most prevalent codes in our data. During the scientific review phase, each student paper was assigned to three reviewers who provided detailed feedback about the science and communication of the project. These reviews were summarized and sent to the students; students were then asked to revise their papers according to the comments. While students may initially have had various levels of understanding of this process, this was the first time that they had actually participated in such a process, and thus many expressed surprise at how much was involved in the revision stage of publication. For example, Davis expressed his surprise at the number of times he was asked to revise: “At that point it was the first experience I had with ... I think I was shocked with how many times I got it back with more corrections.” And Ishita shared something similar, noting how much work was entailed beyond the initial manuscript development:

I didn’t realize how much back and forth there was going to be when I first submitted it, so I can’t honestly say that I was looking forward to learning from this whole experience because then I didn’t know it was going to be this much work.

Michael also expressed a similar feeling of surprise about how thoroughly a manuscript might develop beyond initial submission:

We had always known the stages, like, first you plan, then you make it into a paper, then you submit it, and then it comes back and you review it, and then you change some things, and then you submit it again and it comes back, and then you change some things, and then it probably gets admitted. But I really hadn’t known that there would be that many things to change.

Several other students also spoke of the challenges of addressing the suggested revisions, such as Caroline who shared that “writing the first manuscript wasn’t too hard, but then revising it actually took about a year between the initial submission and actually publishing.”

While students expressed shock and perhaps frustration with addressing the reviewer comments, within this same code a few students also suggested that the scientific review process requires perseverance and is something scientists typically experience. For example, in response to the question about the hardest part of the publication process, Michelle explained: “I would say I guess the early thing, the perseverance. Trying to make sure that it did go through all the stages of revision. And, yeah, that long process.” And Ishita expressed the idea that this revision process is what other scientists experience: “Going through a publication process makes you look at every published paper differently now. I can look at another paper and I’m like, oh, wow, they went through these revisions too.”

Though many students commented on the length of the publication process and the difficulties of revision, they also commented on how the feedback encouraged them to see beyond the scientific content to also focus on their stylistic choices. Students expressed an understanding that primary literature has an intentional structure and style to convey the science effectively as captured in the code “Scientific publication requires clear and effective communication.” For example, Anvika noted a newfound realization of the importance of figures within scientific communication:

I don’t think we realized how much attention people paid to our graphs and figures until we got those comments. So, that was really informative … It taught me to use those in the most efficient way possible … and the power that they can have to get your point across.

Michelle explained that she became more aware of how she presents information: “Yeah I guess it made me more mindful and more conscious of the way that I present the information ... [it] stressed the importance to me of how I present the information.” Caroline, too, indicated an awareness of how she presents information, and specifically that scientific writing should be precise to avoid misinterpretation: “That particular comment about my writing ... not to imply the wrong thing—that was a good thing to point out.” While the comments from Anvika, Michelle, and Caroline hint at audience and stylistic awareness, Ishita expressed an advanced attention to external audiences’ needs while writing:

How clearly you need to communicate your research because, after all, it’s … the public is viewing this and they need the appropriate background and context to understand everything. So, I think it just taught me and showed me the level of attention we need to put in and how much detail there needs to be in a paper like this.

Interestingly, Ishita seems to have extended this experience to future research projects and earlier stages in her inquiry:

When I approach a new research topic or question in these past 2 years, I look at it from an eye of a published author now. When I’m doing my experiment, I’m like, okay, how am I writing it in my paper for it to be effectively communicated?

Michelle similarly expressed extending what she learned about effective communication to other scientific projects:

It’s not only about the science you do, but it’s the way you present it and make it clear to the public. And so, in participating in other science fairs, later on I’ve just been sure to be clear in my presentation.

These comments reveal that students recognized the importance of clear communication and the structures of a science paper as part of reporting their work. Promisingly, a few students connected the purpose of clear communication to the intended audience.

As students discussed the process of publishing their paper, many framed the experience as a cooperative endeavor, discussing the help and collaboration involved in writing and revising the paper in the code “Publication is a cooperative effort.” Many students noted the assistance of a teacher, parent or other student. For example, Anvika explained how she and her coauthor worked together:

We kind of separated sections of the paper when we were writing it. So, I would say that I did a little bit more of the writing, but he also did all of the graphs and presentations of the data analysis that went in our paper.

Advik received help from multiple sources: “We got suggestions from my mother’s friend, which was ... he was a material scientist, and also got some feedback from my father and my mother.” Noah, too, received help from a family member: “Um, my sister looked at it. She told me I need to work on my grammar cause I’m not the best.”

Students generally perceived the reviewers as part of the cooperative effort in publication. Ishita explained how helpful she found the reviewers’ guidance about both her manuscript and the publication process:

The most helpful thing would be, like, how much they guided you through every step of the way, by whether it’d be doing your revisions, or they were really thorough about telling you what to change. Or getting back to you or seeing what the next step will be.

Davis described how he felt included in many exchanges with reviewers:

The reviewers, they were so responsive. We would exchange emails all the time. I was involved in the levels of just, you know, cc-ed on every email to my mentor. I was fully involved in the problem too. I liked that.

And Michelle explained her surprise about the thoroughness of the reviews and how much she learned from them:

I was surprised in a good way how much they … how in-depth the comments were. And so, I was really happy that they, you know, took the time to read the entire article and took the time to give thorough comments... I enjoyed learning new things from them.

Although these students may have been frustrated by the length of the revision process, their comments regarding the reviewers suggest an appreciation for the feedback and a sentiment that the reviewers were working with them in a collaborative manner. Whether these students extend the collaborative nature of the review process to the general endeavor of science is unknown.

Student perceptions on the outcome and value of publication

We found three codes that connected to student perceptions of the outcome or value of publication.

“Scientific publication results in better science and scientists”.

“Scientific publication can progress science and society”.

“Confidence and self-efficacy in pursuing STEM”.

Of these codes, the first two are related to the value of scientific publication within a larger context of doing science; the last code reflects students’ beliefs on the impact of publication in their personal lives or being.

Within the code “Scientific publication results in better science and scientists,” students discussed the outcome of the publication process in terms of their final product and their own scientific growth. Advik explains how reviewer feedback improved his paper, even though he initially thought his paper was quite good:

When we submitted our first publication, we thought everything was perfect, but you give us quite a lot of reviews which had lots of feedback and it was really helpful to improve my publication.

Unlike Advik, Vivek was uncertain of his initial manuscript submission, but also mentioned how the revisions improved his publication:

When I first submitted my manuscript, I wasn’t really sure how good it was. . . But after fixing all those revisions and things like that, reading through my manuscript again, it’s like, wow. I changed a lot. It’s a lot better now.

Caroline appeared to indicate that she believes that revision resulted in a superior scientific product: “I think the feedback on the science and the experimental design just helped me to see, you know, there’s more than one way to do this. There’s a ton of different ways to improve.” Nastia similarly commented: “All the comments were super, just so definitely valid, and I really, really appreciated all the feedback to help improve the study because they were just so in-depth, thorough.” And Jane explained that the reviewer feedback helped improve their methodology:

That was one of the parts that I was really a bit worried about, was when we got really great feedback, but it informed us to go back into our code and to tweak it a bit, and I was worried it would change all of our results and we would have to pretty much change our entire paper. But after doing so, I just think it made our paper so much more stronger.

Beyond producing an improved study or publication, students also commented on how the process improved their own understanding of their science. Vivek shared that the feedback helped him learn more about a biomarker in his study: “And it had a lot of science comments on the biomarker that I had and that actually taught me a lot more about it, my own research.” Anvika similarly expressed the idea that she learned about the science of her own project: “Through that process, you learn your own material a lot better.” And Ishita: “That forced me to look into it, actually research, and then understand, and then eventually put it into my manuscript.” Other students explained how the review process helped develop their scientific skills. Ishita perceived improving as a scientist: “It also helped me grow a lot as a scientist because it just … it gave me an entirely new different experience.” Nastia explained: “Just the honest feedback that I was receiving to just help me as a person to improve my scientific writing.” Similar to what professional scientists conveyed, these students expressed the complex idea that the review process can expand their conception of their project, improve the publication, and develop themselves as scientists (Yore et al., 2004). However, also similar to professional scientists, these students avoided language that connected the review process to the larger process of constructing knowledge.

While all students commented on the benefit of the review process to their own projects, only a few students expressed a more advanced view of publication as a way to advance science and serve society, a sentiment captured in the code “Scientific publication can progress science and society.” Vivek, for example, described how published papers can inspire other scientists:

The publishing aspect is really important because not only will it show other people what I’ve done, but it can also inspire them or it can also encourage them to think more about what I did. And then might even lead to them doing their own investigation.

Nastia indicated that she believed publishing her work could help others:

I really thought that what I found was important and I really wanted people to know about just the negative effects overusing [objects], which is what my study’s about. And so, yeah, I thought that publishing it would be a great way to do that.

Jane had a similar sentiment about the impact of her science in society: “Going through this process, I realized kids like us, they can completely change the way people think with just a computer simulation and a paper.” Although this code was not particularly prevalent, it indicates that some students hold an advanced perspective of publication as a step in the research process for other scientists, or as a way to have a greater impact in the world. However, the two codes together suggest that the majority of students possess a narrower viewpoint that the publication is one that is primarily a personal outcome.

As we probed the student experience, we discovered many comments that implied engaging in publication impacted students’ perceived confidence and desire to pursue STEM, suggesting that increased self-confidence may be another outcome of participating in the publication process for these students. Other research has shown that confidence and self-efficacy are connected to persistence in STEM and thus we were interested in examining these comments (Carpi, Ronan, Falconer, & Lents, 2017; Zeldin, Britner, & Pajares, 2008). Comments in the code “Confidence and self-efficacy in pursuing STEM” support that many students felt pride in their accomplishments and abilities. For example, Michelle explained that the experience:

boosted my confidence . . . now, I still like to read scientific papers, mostly submitted by [organization], and I read them differently now. I can look at the graphs and understand them a lot better than I have before, and I like seeing how other people approach their problems and then relating it to our paper. I think it’s really really cool.

Anvika clearly expressed her pride: “But that I could, you know take an idea that I had and turn it into something that I could put out into the world. I just, I still think that’s the coolest thing I did in high school.” In addition to expressing confidence in their work, students also mentioned that the experience gave them motivation to pursue research in the future. Anvika told us “It definitely kind of sparked my interest in research in general and pursuing science in the long run.” Caroline explained how the experience has impacted her: “Publishing something as a 14-year-old or something like that, that’s pretty impressive. So, I think being able to say that just gave me more confidence to do things that sound pretty crazy.” Michael explicitly expressed a new-found identity as a scientist: “Yeah I always used to think, oh, I’ll be a scientist when I go to college … but now I’m thinking I’m a scientist now once I published a paper.” Altogether these sentiments demonstrate that while the process was seemingly challenging and required work to revise and publish, ultimately the students perceived greater confidence and have not been dissuaded from future scientific pursuits. Although these students may be highly motivated to pursue science regardless of the publication experience, it provides support that engaging in an authentic experience like this, and interacting with professional scientists, may help build scientific identity.

Discussion

Our findings align with that of others, and demonstrate that engagement in authentic inquiry has benefits, but still requires explicit reflective approaches in order for students to develop a functional understanding of how scientists do their work (Lederman et al., 2013; Schwartz et al., 2004). Our study provides three key findings summarized below.

Students hold informed views on the nature of conclusions

Prior evidence has shown that middle and high school students struggle to clearly understand scientific inquiry as how scientific knowledge is developed (Lederman et al., 2019). Although the interviews did not precisely probe student understanding of each tenet of the nature of scientific inquiry, one code that emerged provides evidence that these students hold informed views on at least one aspect of scientific inquiry. The code “scientific research results in nuanced conclusions which may not be conclusive” reveals that students have, or gained, an understanding of the complex relationship between data and conclusions. Some students were quite explicit that the interpretations of their data did not necessarily constitute evidence or proof, but rather support for the hypothesis or conclusion. Some students, such as Anvika, suggested that their conclusions offer the next hypothesis. Davis explained that interpreting data is still challenging. These student views are aligned with the professional view of scientific inquiry articulated by Schwartz, Lederman, and Lederman (2008):

Data and evidence serve different purposes and come from different sources. Data are observations scientists gather during the course of an investigation … Evidence is a product of data analysis and interpretation. (p. 5).

Students like Noah and Rosa also discussed carefully hedging their conclusions to avoid overstating the meaning of their results. Thus, these students appear to understand the inquiry principle set forth by Lederman et al. (2019) that students should know that “research conclusions must be consistent with the data collected” (p. 491). These results are promising because research has demonstrated that students in the United States particularly struggle with the nature of conclusions and the relationship between data, evidence and conclusions (Lederman et al., 2019). We cannot conclusively determine whether these are pre-existing understandings or whether students developed these ideas as a result of their publication experience. However, many of the students made these comments in response to questions directly related to the review process. For example, in response to a question about what he learned from reviewers about his discussion, Davis explicitly stated his new learning regarding conclusions:

That’s what I learned about making conclusions. I was very willing to expand conclusions, but it was really baby steps that had to be taken instead. That’s what I learned about that section—that I just need to step it back a little bit to make sure.

Similarly, Vivek explicitly mentioned the reviewer feedback impacting his conclusions: “On my own, I came to the conclusion, but the reviewers, they put a lot of feedback on how I should elaborate on that and how I should validate, things like that.” Representative reviewer comments can be found in Supplemental Table 1, which show how reviewers encouraged students to be critical and careful in developing their conclusions. These data do not provide conclusive proof that engaging in publication can develop students’ views of the nature of data and evidence, they do provide an initial clue that engaging in scientific review expanded student understanding of the tentativeness of conclusions.

Students hold under-developed views on the roles of some disciplinary literacy practices, but value the role of scientific review

Analyzing student perceptions of disciplinary literacy with respect to the three interconnected practices of reading, writing and evaluating scientific literature, suggests that, overall, students exhibit underdeveloped views of disciplinary literacy within the enterprise of science. Of the three practices, students in our study expressed the least-developed views in regard to the value and purpose of reading the literature to learn content, develop ideas and construct new knowledge. Others have advanced the image of the ideal scientist who reads the literature for many reasons, including to develop project ideas and hypotheses, revise methodologies, interpret results, and construct relevant conclusions (Tenopir & King, 2004; Yore et al., 2004). We found, however, that students viewed the published literature more naïvely. Interestingly, while all student papers include references in their introductions, students, generally, did not express using literature of any type in the development of their research idea or experimental plan. Instead, most students used familiar resources such as parents and teachers to guide their project development. This suggests that literature may be incorporated superficially in the papers and that students do not view the literature as an important or initial source of observation and idea inspiration. Only one student, Vivek, articulated a more advanced view that published literature is part of the inquiry process and that his paper might be used by other scientists. Yet, even with this advanced view, Vivek did not indicate that he used the literature as part of his own inquiry process. Part of this disconnect could stem from the fact that several students did not intend to pursue publication prior to starting their research projects, and thus the recognition of the value of published literature may have come later in the project timeline. Regardless, our data suggest that these students did not fully engage in the inquiry process as scientists because using the literature was neglected as a mechanism in that process. This reinforces the importance for practitioners and mentors to emphasize and model the use of the literature, whether it be primary literature or popular science articles, in different stages of an inquiry project. While we don’t mean to discourage students from using familiar resources, students could be encouraged to use the literature as a source of observation and idea inspiration, which is increasingly possible given today’s many open-access scientific publications, including those specifically designed for pre-college students.

Students also held underdeveloped views on the role of writing to communicate and stimulate new knowledge construction. Students noted that their work could influence society by informing the public of their findings’ significance and implications. With this perspective, students begin to explore the larger impact of their work, but this also suggests that students primarily view their writing as knowledge-telling. Contrary to the scientists interviewed by Yore et al. (2004), the students we interviewed did not express the concept of writing generating insight or stimulating new ideas. We do not mean to imply that these students did not learn from the writing, but our results reinforce that explicit reflection and discourse may be necessary for students to recognize their own learning and the value of writing as a way of learning within the inquiry process.

Of the three practices of disciplinary literacy, our students held the most informed views on how communal evaluation can strengthen scientific understanding. Scientific inquiry occurs within a community of scientists and this community engagement is quite evident within the peer-review process. The students we interviewed specifically referenced the scientific review as a collaborative process which can improve scientific knowledge construction, as evidenced in the codes “Scientific publication is a cooperative endeavor,” “Scientific publication requires revision,” and “Scientific publication improves science and scientists.” Comments in these codes reveal that students viewed the scientific reviewers as constructive and collaborative, and ultimately that the review and subsequent revision process produced a product and research of higher quality. Students such as Caroline, Nastia, Vivek and Ishita made explicit reference to the feedback improving their research. And two of the students extrapolated their experience to that of professional scientists. These student comments directly reflect the inquiry concepts that science occurs within a community, that “Scientists review and ask questions about the results of others’ work” (NRC, 2000), and that “Communication and peer review impact what and how science progresses” (Schwartz et al., 2008, p. 6). Although it is promising that students recognized the impact of scientific review on the progression of their own science, they did not express the contemporary epistemology that the scientific review process is a mechanism to advance scientific knowledge as a whole. However, it may be too ambitious to expect students to express such an advanced perspective, especially given that professional scientists also struggle to explicitly acknowledge this (Yore et al., 2004). Even a former editor explained his view of peer-review, not as one that aids in the construction of knowledge, but as a process that would “raise the standard of the papers published in the journal” (Tobin, 2002, p. 145). In contrast to some professional scientists, who view reviewer comments as an obstacle or challenge to getting published, these students primarily hold the view that the reviewers are constructive and collaborative, though it remains to be seen whether these students retain these views of the review process after future professional encounters (Yore et al., 2004).

The idea that peer-review can be used as a learning tool in the classroom is not foreign and has been used in a variety of settings with mixed results. Some college educators have used peer-review within the writing-to-learn pedagogy as a way to help students identify and correct misconceptions, learn content, or understand the scientific process (Finkenstaedt-Quinn, Snyder-White, Connor, Gere, & Shultz, 2019; Weaver et al., 2016). Trautman explains that one outcome of engaging in peer-review is learning that science is a revisionist endeavor, and knowledge is established through a continuous process of review and revision (Trautmann, 2009). While our students were not reviewed by peers in the strict sense, our work further supports that participating in peer-review can help students understand science as a communal and revisionist endeavor.

Students are capable of engaging in this type of authentic practice, and it can build perceived self-confidence and self-efficacy in students

As Schwab noted in 1962, engaging pre-college students in the primary literature is a feasible endeavor. These students’ experiences not only support Schwab, but also demonstrate that students may benefit in their confidence and self-efficacy by engaging in such authentic experiences. As students build their confidence in doing scientific inquiry, they will hopefully identify as part of the scientific community, something explicitly mentioned by several students. Sense of belonging to the scientific community and building one’s scientific identity and self-efficacy are correlated with greater retention in the STEM pipeline (Carpi et al., 2017; Robnett, Chemers, & Zurbriggen, 2015). Others have shown that historically underrepresented students in STEM, who engage in authentic research experiences at the undergraduate level, express gains in self-efficacy and identity, and increased interest in high-level STEM careers (Carpi et al., 2017). Thus, the publication process, and the potential gains in self-efficacy and identity, could specifically benefit students from historically underrepresented or disadvantaged backgrounds.

Limitations

Although our findings support the value of engaging students in scientific review and publication, there are several limitations to our research which highlight challenges in implementing disciplinary literacy practices more broadly. First and most clearly, our data set includes a limited number of likely self-motivated student authors. While all students noted the publication process as a positive learning experience, a larger survey of student authors is required to confirm this. Second, as these students appeared to be self-motivated, as this was a voluntary experience, it remains possible that any perceived learning would not be so apparent in less-motivated students. Third, participants ranged in contextual factors of their research project, for example: the type of research environment they had access to and who they selected as mentors for their projects. For example, five out of the 12 participants either consulted with or were directly mentored by professors, another five out of the 12 received parental help, and two were mentored by their teachers (Table 1). Access to university-level labs or professional scientists is not feasible for most students. Broadening access to professional scientists is helpful, and something that the JEI has endeavored to address on a small-scale with an online Ask-A-Scientist component and several other initiatives. However, a larger challenge is extending necessary professional development to teachers so that they may bring disciplinary literacy practices into their classroom. Professional development could help mitigate another challenge, namely variation in teacher content-expertise, mentorship styles, and pedagogical strategies, which may additionally impact student experiences with the publication process. Fourth, and perhaps most importantly, the data set reveals perceived learning of student authors. To understand how scientific review and publication shapes a student’s views of scientific inquiry, it will be necessary to administer a pre- and post-validated survey, such as the Views of Scientific Inquiry to a larger number of student authors (Lederman et al., 2014; Lederman et al., 2019). Problematically, this survey does not include explicit reference to peer-review and publication as components of inquiry. Thus, we first advocate for a re-examination of such assessments in light of the growing data on the importance of integrating peer-review and publication in scientific inquiry. Future research should investigate and compare changes in student author understanding of scientific inquiry, self-efficacy and confidence before and after the scientific review process.

Conclusions and implications

Our findings suggest that the emphasis on publication as a mark of success may be engrained early in the minds of young scientists as they view their publication primarily as a personal achievement. These results further emphasize the important challenge of moving beyond the practice of using communication assignments as finales to research and more fully integrating disciplinary literacy practices within the pre-college scientific inquiry experiences. Although it requires more research, our findings suggest that students developed in their views of science inquiry and their own research as a consequence of interacting with other scientists through the scientific review process. Thus, these findings emphasize the importance and potential benefits of engaging students in authentic disciplinary practices. If a goal of pre-college science education is for students to develop a true understanding of scientific inquiry, this is undermined by neglecting primary literature as part of doing and learning scientific inquiry. This challenge has been recognized in college-level sciences, and it has led to writing-intensive curriculum and the emphasis and use of primary literature in the science classroom (Hoskins, Stevens, & Nehm, 2007; Janick-Buckner, 1997; Kozeracki, Carey, Colicelli, Levis-Fitzgerald, & Grossel, 2006; Muench, 2000). We assert that these approaches can 1. Be modified for use in the high school level classroom; and 2. Require revision such that the literature is not approached as an end-product of science inquiry, but rather an integrated part of the inquiry process. We suggest, as others have, that intentionally including disciplinary literacy practices as mechanisms of inquiry will help students develop a deeper understanding of scientific inquiry and more advanced sense-making practices as emphasized by NGSS (Koomen et al., 2016; NGSS Lead States, 2013; Schwab, 1962). Specifically, as students engage in inquiry-based projects, different types of literature should be encouraged at all stages of the research process and explicit reflection on the purpose of using the literature is critical. A clear hurdle to these proposals is the already numerous standards in place for pre-college science classrooms. A first step to addressing this might be to evaluate the disciplinary literacy practices already in place and determining where authentic texts (such as the primary literature) can be used in place, or in combination with, textbooks.

Future research should explore the impacts of different types of mentors for students, with the goal of better understanding how to equitably support student research, use of primary literature and potential publication. For example, we are specifically interested in understanding how teachers have found ways to integrate the JEI publication process into course-based research and how this integration impacts student engagement in scientific inquiry. Finally, future studies should investigate the changes in learning, science identity, confidence and self-efficacy before and after the publication experience to precisely understand how this process impacts the development and retention of young scientists, and specifically young scientists of historically underrepresented or disadvantaged backgrounds in STEM.

Ultimately, while we understand that using primary literature in every classroom may not be feasible, we advocate for the exploration of these processes and ways to instill the value of the primary literature within pre-college science inquiry.