Previous studies on expert and novice problem-solving strategies were reflected in the design of the formative assessment questions. Specifically, it was hypothesized that conceptual explanation questions would help students think about the questions more conceptually, so that they would start to solve them using scientific concepts and laws. Therefore, short written questions in the tasks asked the students to explain or to justify their answers without using a formula. Nonetheless, when we were interviewing students, we found that they preferred to use equations and mathematical concepts when explaining physical situations. Although the three participating students commonly used equations or mathematical concepts in their explanations, how they used the equations or mathematical concepts differed. Detailed findings are presented below in three subsections representing patterns in problem-solving strategies. Formative assessment Tasks 1, 2, and 3 were designed to address the topic of Motion and Force, while Task 4 covered the topic of Energy Conservation. We analyzed interview data by these two topics. Note that two terms—formula and equation—were not differentiated in the analysis of data; instead, they were considered synonyms.

### Alex’s case – using equations as a conceptual understanding tool

#### Motion and force

When interviewing Alex, we asked him what would happen if a person pushed a box, then let it go (Task 2). He said, “If it is frictionless, the box will move forever with a constant velocity, and if friction exists, the speed will decrease and eventually the box will stop.” This answer was very similar to his original written response. Next, we asked what would happen to the box’s motion after another box was placed on top of it. Alex said, “I don’t know how to explain this without a formula.” Because the original questions had asked students not to use formulas, he assumed that he was not allowed to use one in this explanation, and obviously he was struggling to explain without it. We told him to use formulas whenever he wanted, and he quickly jumped into using one.

Alex: Resultant force equals mass times acceleration, so if you have a bigger mass. Uh, if the resultant force was 50N, that’s the force you applied, and then you had 10N in friction, for example, then the resultant force is 40. You had, if you had 20kg, the acceleration would be 2. If you had 50kg, the acceleration would be 4 over 5, which is 0.8, which is less than 2. So, the more mass you have the smaller the acceleration is going to be, as a result of the resultant *force* equals *ma* equation.

In this statement, Alex explained what would happen in the given situation with algebraic solutions, using *F = ma* equation, and concluded that mass would affect the object’s acceleration, as he demonstrated. He further described how the eq. (*F* = *ma*) helped him to explain the given physical situation.

Alex: If you use the formula, then it makes it much easier, because in real life, you never see something moving without friction, so it just clouds your judgment a bit.

In this statement, Alex described the role of equation for him as a conceptual understanding tool, especially in an ideal situation that is not observable in real life. This was something the author had not initially expected from the students during their interviews. When we asked Alex the next question in Task 3, his answer further supported the finding that equations helped him understand physical situations. Specifically, we asked, in a situation when a person was pushing a cabinet on either a frictionless or a frictional surface, what would happen to the cabinet’s motion and why.

Alex: The normal force is, the gravitational force cancels out the *y*, so the only thing acting on the—in the *x*-direction, which is the direction being pushed is the applied force, so as small of a force you apply to it, it’s still going to move it because there’s nothing opposing it…if there was friction, I agree that it won’t move. Because the friction, the friction is the coefficient of friction times the normal force, so, since it’s a really big object, it’s going to have a significant amount of friction acting on it.

In his verbal explanation, Alex used a mathematical concept and an equation to explain the given phenomenon, using the vector concept for two components of force and a mathematical equation for frictional force. Obviously, he found equations useful to make sense of physical situations and to explain his understanding to others. Notably, he started his answer by referring to the formula for kinetic friction force and used the formula as a tool to explain why the cabinet wouldn’t move on a frictional surface. His explanation again demonstrated that equations and mathematical concepts were useful to understanding and interpreting scientific phenomena, and not only as a simple computational tool, at least for Alex.

#### Conservation of energy

Task 4 was designed to investigate students’ conceptions of mechanical energy and its conservation. We asked Alex, when a skater is skateboarding on a track with no friction, what would happen to the skater’s highest speed as the skater’s mass increases? He again asked us if he could use equations. We confirmed that he was allowed to use equations anytime he wanted. Then he immediately started writing equations on the board (see Fig. 2). While he was writing, he explained each variable involved in the equations:

Alex: So, her initial, so, um, at the start, her initial energy is *mgh* + ½ *mv*_{0}^{2} and then her final [writing on board] *mgh* + ½ *mv*_{f}^{2}, but the smaller thing to do is that they [mass] all cancel out, so the mass is really, it doesn’t play a role in the height or the velocity. And then, if you wanted to see how the conversion of energy works, if you were initially starting at the maximum height, whatever that is, you could do *½ mv*^{2}. At the start, her velocity is 0, at the top, so this cancels out, if we’re analyzing it at the bottom, which is her max speed, then this [*h*] is 0, and then you just do *gh = v*^{2}. To find her velocity. Just looking at this, there’s no mass in this, so it doesn’t matter [the skater’s speed]. When you actually work it out, all the masses cancel out, so it doesn’t matter what the mass is, in reality, when you actually calculate it.

This response was different than his original written response to the same question: “If the skater has a larger mass, she will in turn have a larger gravitational potential energy since GPE [gravitational potential energy] has a direct relationship to mass. As a result and according to the principles of conservation of energy, the KE [kinetic energy] will be greater and thus the velocity will be greater.” In this original written response, Alex included a typical misconception that heavier objects fall faster (e.g., Gunstone, Champagne, & Klopfer, 1981; Lazonder & Ehrenhard, 2014); “If the skater has a larger mass…thus the velocity will be greater” (in his written response). This was the only case of a misconception found in Alex’s written responses. Notably, when he was using equations, he deduced that “it doesn’t matter what the mass is, in reality, when you actually calculate it” from his step-by-step problem-solving procedure using algebraic solutions. Although he solved the problem using equations through algebraic computation, he explained how the object’s velocity and height would change as the object moved: “At the start, her velocity is 0, at the top, so this cancels out, if we’re analyzing it at the bottom, which is her max speed, then this [*h*] is 0, and then you just do *gh = v*^{2}.” Then he connected conceptual meaning to the equation: “Just looking at this, there’s no mass in this, so it doesn’t matter [the skater’s speed].” This confirmed that for Alex, equations were the first tool to make sense of physical situation. In other words, when he applied an equation to a physical situation, he considered variables related to specific situations, then connected conceptual meaning to the variables, which indicated that for him, equations played a role in analyzing and understanding physical situation.

### Christopher’s case – using equations as an explanatory tool

#### Motion and force

We asked Christopher a question—which tank shell would go farther when the initial angles for two tank shells were different (Task 1). In his original written response, he mentioned that “tank A (initial angle: 45 degree)’s speed is broken up more evenly and this results in more air time which leads to more distance covered in the x axis as well.” This answer was similar to Christopher’s thinking-aloud response, so we asked him to elaborate on what he meant by “speed is broken up more evenly.” Below is his response.

Christopher: Because the velocity is a vector quantity, the speed is still the same, but the velocity, the x and y axis are going to be more evenly split [for Tank A, with a 45-degree initial angle], whereas for Tank B [10-degree initial angle] it would have been almost all in the *x* axis and close to none in the *y*, so it wouldn’t get that much air time because the force of gravity still stays the same.

As seen in his response, Christopher deduced his answer from a mathematical concept (vector in this case) explaining why the 45-degree shell would have a greater horizontal range than the 10-degree shell one. His problem-solving strategy in the next questions (questions from Tasks 2 and 3) further confirmed that he used mathematical concepts and equations to explain physical situations. For example, when asked to compare two situations from Task 2—a person pushes a box and lets it go, and after placing another box on top of that, a person pushes both boxes and lets them go—Christopher immediately used *F = ma* and explained the situation.

Christopher: The velocity and the speed will be decreased because, when applying force, force is mass times acceleration. So, if it would be the same exact force with a higher mass, then the acceleration would have to go down significantly in order to keep the same number [force]. So, because of this, it wouldn’t speed up as much, so it would have a lower velocity after the force was applied [compared to the previous situation]. While you are pushing, the acceleration is constant. And if they let it go, there is no acceleration. Then speed will stay the same.

In his statement, he referred to *F = ma*, and explained why the box’s acceleration would be smaller when its mass increased using algebraic solutions, which is similar to Alex’s case. The difference is that Christopher’s explanation contained an interpretation of the relationship among velocity, acceleration, and applied force: “So, because of this, it wouldn’t speed up as much, so it would have a lower velocity after the force was applied.” This implies that Christopher did not just use the equation as a computational tool, but linked meanings to variables (force, velocity, mass, and acceleration) and interpreted a relationship among them. When we asked him a question from Task 3—when a person is pushing a cabinet, how will the cabinet’s velocity change after passing over the frictionless surface and traveling onto the surface with friction?—his answer reconfirmed that he considered the relationship among variables and gave conceptual meaning not only to the variables but also to the relationship, and used a mathematical concept as an important tool to interpret a physical situation.

Christopher: So, the velocity is 100% dependent on the acceleration, which depends on the force, and then in this scenario, it is the force at first, it has a much higher total net force in the *x* direction, whereas later on it decreases [on a frictional surface], but there’s still a positive net force in the *x* direction, so it will continue. The reason why it continues to speed up is because the acceleration is still positive. ‘Cause mass can’t really be negative so that [acceleration] is the only variable [to determine the change of velocity]. So, that’s why velocity continues to increase, it’s just not as much as before.

In his statement, Christopher did not interpret an individual variable separately; rather, he first considered the relationship between force, velocity, and acceleration using the concept of vector and scalar quantity (e.g., mass is not a vector quantity), and explained how each variable was influenced by the other variables’ changes. From the statements above, it is clear that Christopher reasoned through a physical process by interpreting relationships among variables and attaching conceptual meaning to the relationship and the variables.

#### Conservation of energy

When we asked Christopher about change in the skater’s highest speed when the skater’s mass increased, his original written and oral responses contained the common answer that the skater’s highest speed would stay the same because gravity acts on all objects equally: “the downward acceleration will be the same.” We further asked him about how total mechanical energy changes. His response is below.

Christopher: Her [the skater’s] mechanical energy would increase because the velocity would stay the same for kinetic, but the mass would go up, so it would make the answer higher. And it’s probably easier to think of it with GPE, can I use the formula to it?

Then he drew a formula on board (Fig. 3), and explained why the total mechanical energy would change.

Christopher: This is mg. Since these two [*gh, ½*] stay the same for both cases, they can be canceled out. So then, these are the only variables in ME (mechanical energy), so if this [*m*] increases, then the whole system[’s energy] will increase, but it won’t change this [*v*] in the specific scenario. If you were to use the equations, once you were to set them equal to each other and solve for the final answer for each, they would still be the same, even though the mass is higher. But because it’s multiplied, you can cancel it [*m*] on both sides for that specific scenario, so it mainly just depends on the constant *½* and then the variable of height and the final velocity which would be the same for this case.

In his response, Christopher first explained the physical situation using the concept of energy and considered the situation as a system: “Her [the skater’s] mechanical energy would increase,” and “so if this [*m*] increases, then the whole system[‘s energy] will increase.” In order to prove why mass doesn’t affect the skater’s speed, he used an equation as an explanatory tool—“And it’s probably easier to think of it with GPE, can I use the formula to it?”—and showed that mass doesn’t affect the skater’s speed: “You can cancel it [*m*] on both sides for that specific scenario.” A noticeable difference from Alex’s approach is that Christopher used equations to prove his claim and to explain it in an easier way, while Alex used equations to make sense of the situation. In other words, equations were in play mainly as explanatory tools for Christopher, whereas they acted as conceptual understanding tools for Alex. Similarly to his previous responses to questions in Motion and Force, Christopher again demonstrated that he considered how all variables were related each other in the system, and attached meaning to the relationship and variables. Interestingly, he often used the phrase “specific scenario,” so we asked what it meant. Below is his response.

Christopher: The equations don’t really help because even though I see it and it’s in my head, but it’s not really useful if I don’t know the scenario. If it’s some problems, I know, are purposefully shaped to muddle it up, and make it purposefully confusing, but usually, when you run the scenario, in a program or in your head, it kind of takes out that confusing stuff.

The above response illustrated that Christopher conceptually interpreted the physical situation first, then translated equations into the physical situation. This strategy shared a commonality with Alex’s in that both students used equations in their explanations and connected how variables in the equations changed as the specific physical situation changed. At the same time, there was a difference between the two students. Christopher’s strategy started with an analysis of the situation, creating a physical scenario and then translating equations into the physical situation, while Alex mentioned relevant equations first, then connected them to the physical situation.

### Blake’s case – using equations as a computational tool

#### Motion and force

When we asked Blake which one would go farther when shot from a cannon, a tank shell or a baseball (when air resistance was negligible; Task 1), her original written response and her thinking-aloud response were similar: the mass of an object is not relative to its motion. When we asked her to explain why, she said:

Blake: Because I don’t see *kg* on the units at all [in the simulation]. *kg* is the unit for mass, kilograms, so, it’s not written as *kg/m/s* or something. You could easily compare it with units and mass is not part of the unit.

Her response was interesting in that she used the unit of velocity rather than acceleration. Also, she did not show her conceptual understanding of physical variables and their relationship as Christopher had done. We further asked her what factors should be changed to maximize the horizontal range of the projectile object, in order to elicit her reasoning about a projectile motion. Below is her response.

Blake: You need to throw it faster. Um, because, if you look at gun for example. It’s a really high velocity. So, you just see it going like straight because it’s just high velocity. And, um, if, if I’m throwing this phone, maximum distance it could go is like here [tosses phone, not very far]. Angle? I think…like the maximum distance for x axis and y axis is 45 degrees, but I think it should be a little lower. Around 45 but plus or minus 5 degrees, so like 40 degrees.

Interviewer: Why would you say that?

Blake: It doesn’t get that much time for vertical velocity, but the horizontal velocity will be faster.

In her response, Blake used real-life examples—shooting a gun and throwing a phone—as analogies to reason how to increase the horizontal range of a projectile object. However, when she threw the phone, she tossed it, which started it with a different initial angle from that of a bullet shot from a gun*:* “You just see it going like straight because it’s just high velocity.” Although she considered two directions of velocity when determining the optimal initial angle, she did not provide a scientifically reasonable explanation for why the initial angle should be lower than 45 degrees. It might be that Blake had learned that 45 degrees is the angle used to maximize range, but that she thought velocity would be more critical than the angle to determine the range, especially that the *x*-component of velocity would more important than the *y*-component because an object will fly faster horizontally than vertically when the *x-*component is greater. Thus, she lowered the initial angle a little bit. In the above statements, Blake did not demonstrate that she could consider the relationship between variables and link conceptual meanings to them (e.g., “Because I don’t see *kg* on the units at all” and “It doesn’t get that much time for vertical velocity, but the horizontal velocity will be faster”).

For the next question, we asked what would happen to the box’s motion after another box was placed on top of it. She said, “It would still be constant and stay at constant velocity in that motion.” We asked the question again, to clarify if she understood it.

Blake: Yeah. The velocity would be the same. After you let it go. So it will be at constant speed. And the force is proportional to the...wait, well acceleration is proportional to force and mass.

In her response, Blake attempted to apply Newton’s second law (*F = ma*), as the other two students had; however, she didn’t realize that acceleration is inversely proportional to mass, and therefore the velocity would be changed by the different acceleration. As a result, her response involved a misconception that mass doesn’t affect the speed of an object. In other words, she demonstrated her lack of understanding of the relationships between the variables (acceleration, velocity, mass, and force) involved in the situation. Her response to the questions confirmed that she explained scientific phenomena using variables in equations but failed to recognize the relationships among them. Instead she focused on individual variables, e.g., how acceleration will change as force changes, but did not explain how that would change velocity. She also did not explain how two components of velocity affect an object’s motion. Interestingly, she also used the unit of variable to justify her answer without applying conceptual meanings to it. For Blake, equations and units seemed to play important roles in explaining physical situations, but her connection of equations to physical situations was, at best, based on interpretations of individual variables.

#### Conservation of energy

When we asked Blake about change in the skater’s highest speed when the skater’s mass increased, her original written response was that her highest speed would increase because the mass of the skater would require more energy. When we interviewed her, her answer was different from her original response.

Blake: I think it should stay the same. I was thinking of the formula.

When we asked her to explain in more detail, she wrote an equation on the board (Fig. 4) and explained what it meant.

Blake: The highest point, because there won’t be any kinetic energy. And it’ll be *mgh*. Also ½ *mv*^{2} and it [*m*] cancels out. It was exactly the same. The speed was the same. But—wasn’t there a bar graph [in the simulation]? Well, the total energy was bigger [in the simulation]. The total energy. But the total energy was same—no bigger.

Similarly to Alex, Blake used an equation to explain that the skater’s speed wouldn’t change because *v* doesn’t contain *m* after canceling out. However, she did not describe why kinetic energy is zero at the highest point and why potential energy is zero at the bottom. It might be that she just did not mention this, but it was obvious that she did not understand how the object’s mass affected the system: “But the total energy was same—no bigger.” We further asked her how the total mechanical energy of the skater would change when the skater’s mass increased. This time, she said, “Well, the total energy was bigger. ‘Cause energy depends on mass and either height or speed of a person.” As seen in the response, she thought of variables in equations of gravitational potential energy (*mgh*) and kinetic energy (\( \frac{1}{2} \) *mv*^{2}). When asked why she previously had said the total mechanical energy would be the same, she answered, “because energy is always conserved.” This illustrated her misconception that the amount of energy should always be the same regardless of mass; however, when she considered variables in equations of PE and KE, she answered the question accurately. Throughout the interview, we found that Blake’s strategy to solve questions was consistent across different tasks; she used formulas and units as her first approach. However, a difference between Blake and the other two students is that although she used equations and variables, she did not explain how the variables influenced each other; and how they would change as a specific situation changed. In other words, she did not translate equations into physical situations nor link conceptual meanings to the variables and the relationships between them. The findings showed that for Blake, equations were more likely used as a simple computational tool.

### Disconnection of students’ problem-solving strategies from physics lecture

The three students all mentioned that they liked simulation-based questions. Alex said that the questions themselves made him think a lot, and running simulations also made him think more deeply: “Beforehand it [the task] just seems really simple, so you don’t put much thought into it. That’s easy. Just write it down, but then, once you run it, it makes you think about it more. So that’s cool too.”

As seen in his responses to the questions in the tasks, Alex used equations as conceptual understanding tools consistently across tasks. When we asked if he had learned this approach from his physics course, he said that his physics class heavily focused on solving problems but mostly by just reading off equations and plugging in numbers.

Alex: Physics is not about reading equations and stuff off a slide. It’s about working things by hand, and my professor, he has all the solutions to the problems in the book. He had them on a clear sheet of paper, and a Sharpie and then, so if he has problem 20, he puts problem 20 on the projector, and then he put that clean sheet there, and then he points to, oh here I did “v = a + blah blah,” so that’s really not effective at all in my opinion.

Christopher mentioned that the formative assessment would be very helpful for a lot of students, because it showed physical scenario. His physics class was more formula-based, with activities such as showing a formula and plugging in numbers to demonstrate how to solve a physics problem, which Christopher felt was disconnected from how he learned science. As he demonstrated, he learned best when he created a physical scenario, then translated it into equations. Alex also mentioned that physics is “about working things by hand,” which implies that he emphasized linking problem-solving procedures to physical situations. In Blake’s case, she mentioned that “It will help students to learn the concept better, but I think students will hate it [the formative assessment] because students will be like, ‘I don’t have time for this. It’s just that I am like too busy for this.’” In sum, the three students had a common opinion that the simulation-based formative assessment had helped them understand the given physical situation better, but the reasons why they liked it differed, as did their problem-solving strategies.